Question

By May 2020, will a single language model obtain an average score equal to or greater than 90% on the SuperGLUE benchmark?

Resolved :No

28 comments28

Total Forecasters74

Community Prediction85%

Make a Prediction

50%

community: 85%

Did this actually happen?No

Community Baseline Score

-125.7

Community Peer Score

-14.8

median 85.0%mean 82.0%

Authors:

ghabs

Opened:

Closes:

Resolves:

AI Technical Benchmarks

Computing and Math

Artificial Intelligence

Machine learning

Linguistics

On January 1, 2025, which frontier AI lab will have a publicly available model with the highest score on the MMLU benchmark?

What will be the best score on the GAIA benchmark before 2025?

47.6

What will be the best non-human SAT-style score on the hard subset of the QuALITY dataset by January 1, 2030?

96.6

Comments

? comments

Authors:

ghabs

Opened:

Closes:

Resolves:

AI Technical Benchmarks

Computing and Math

Artificial Intelligence

Machine learning

Linguistics

Sam Altman said he 'sometimes' used ChatGPT to answer Reddit AMA questions

Business Insider•Nov 4, 2024

AI, Human Language, and US Presidential Elections

Psychology Today•Nov 4, 2024

The Gap Between Open and Closed AI Models Might Be Shrinking. Here’s Why That Matters

Time•Nov 5, 2024

Learn more about Metaculus NewsMatch

On January 1, 2025, which frontier AI lab will have a publicly available model with the highest score on the MMLU benchmark?

What will be the best score on the GAIA benchmark before 2025?

47.6

What will be the best non-human SAT-style score on the hard subset of the QuALITY dataset by January 1, 2030?

96.6