Question

When will AI achieve competency on multi-choice questions across diverse fields of expertise (MMLU)?

27 comments27

Total Forecasters86

Community Prediction2025-07-12

Make a Prediction

My Prediction

community

lower 25%

median

upper 75%

>2039-12-31

...

2024-10-02 17:25

2025-07-12 18:12

2026-10-01 17:04

1.3%

Authors:

isinlor

Opened:

Closes:

Scheduled resolution:

AI Demonstrations

Computing and Math

Artificial Intelligence

Machine learning

Artificial intelligence

Computer science

On January 1, 2025, which frontier AI lab will have a publicly available model with the highest score on the MMLU benchmark?

What will be state-of-the-art accuracy on the Massive Multitask dataset on the following dates?

94.9

When will an AI be able to solve over 80% of competition-level coding problems on its first try?

2027-04-13

Comments

? comments

Authors:

isinlor

Opened:

Closes:

Scheduled resolution:

AI Demonstrations

Computing and Math

Artificial Intelligence

Machine learning

Artificial intelligence

Computer science

When Will AI Take Over the World? We Asked ChatGPT

Newsweek•Oct 10, 2024

Researchers question AI’s ‘reasoning’ ability as models stumble on math problems with trivial changes

TechCrunch•Oct 12, 2024

Here's how Anthropic CEO Dario Amodei defines artificial general intelligence

Business Insider•Oct 13, 2024

Learn more about Metaculus NewsMatch

On January 1, 2025, which frontier AI lab will have a publicly available model with the highest score on the MMLU benchmark?

What will be state-of-the-art accuracy on the Massive Multitask dataset on the following dates?

94.9

When will an AI be able to solve over 80% of competition-level coding problems on its first try?

2027-04-13