Question

When will AI achieve competency on multi-choice questions across diverse fields of expertise (MMLU)?

27 comments27

Total Forecasters86

Community Prediction2025-07-10

Make a Prediction

My Prediction

community

lower 25%

median

upper 75%

>2039-12-31

...

2024-10-18 10:29

2025-07-10 16:37

2026-09-21 10:44

1.2%

Authors:

isinlor

Opened:

Closes:

Scheduled resolution:

AI Demonstrations

Computing and Math

Artificial Intelligence

Machine learning

On January 1, 2025, which frontier AI lab will have a publicly available model with the highest score on the MMLU benchmark?

What will be state-of-the-art accuracy on the Massive Multitask dataset on the following dates?

94.7

When will an AI be able to solve over 80% of competition-level coding problems on its first try?

2027-02-22

Comments

? comments

Authors:

isinlor

Opened:

Closes:

Scheduled resolution:

AI Demonstrations

Computing and Math

Artificial Intelligence

Machine learning

OpenAI is reportedly struggling to improve its next big AI model. It's a warning for the entire AI industry.

Business Insider•Nov 12, 2024

Google’s AI ‘learning companion’ takes chatbot answers a step further

Verge•Nov 12, 2024

How Synergistic Is the Combo of AI and Humans?

Psychology Today•Nov 13, 2024

Learn more about Metaculus NewsMatch

On January 1, 2025, which frontier AI lab will have a publicly available model with the highest score on the MMLU benchmark?

What will be state-of-the-art accuracy on the Massive Multitask dataset on the following dates?

94.7

When will an AI be able to solve over 80% of competition-level coding problems on its first try?

2027-02-22