Question

What MMLU benchmark score will Google DeepMind's Gemini model have on release?

Resolved :71.8

16 comments16

Total Forecasters19

Community Prediction74.4

Make a Prediction

My Prediction

community

lower 25%

median

upper 75%

...

71.85

74.45

84.21

What was the final result?71.8

Community Baseline Score

-18.2

Community Peer Score

0.8

Authors:

TNTOutburst

Opened:

Closes:

Resolves:

Computing and Math

Artificial Intelligence

On January 1, 2025, which frontier AI lab will have a publicly available model with the highest score on the MMLU benchmark?

What will be the best score on the GAIA benchmark before 2025?

47.6

When will AI achieve competency on multi-choice questions across diverse fields of expertise (MMLU)?

2025-07-10

Comments

? comments

Authors:

TNTOutburst

Opened:

Closes:

Resolves:

Computing and Math

Artificial Intelligence

OpenAI is reportedly struggling to improve its next big AI model. It's a warning for the entire AI industry.

Business Insider•Nov 12, 2024

Google’s AI ‘learning companion’ takes chatbot answers a step further

Verge•Nov 12, 2024

Google’s Gemini AI now has its own iPhone app

Verge•Nov 15, 2024

Learn more about Metaculus NewsMatch

On January 1, 2025, which frontier AI lab will have a publicly available model with the highest score on the MMLU benchmark?

What will be the best score on the GAIA benchmark before 2025?

47.6

When will AI achieve competency on multi-choice questions across diverse fields of expertise (MMLU)?

2025-07-10