Question

When will a language model meet or exceed the human baseline on SuperGLUE?

Resolved :Dec 30, 2020

6 comments6

Total Forecasters37

Community Prediction2021-03-11

Make a Prediction

My Prediction

community

lower 25%

median

upper 75%

>2024-07-21

...

2020-12-26 09:39

2021-03-11 14:59

2021-06-28 07:42

0.4%

What was the final result?Dec 30, 2020

Community Baseline Score

90.9

Community Peer Score

46.6

Authors:

Matthew_Barnett

Opened:

Closes:

Resolves:

AI Technical Benchmarks

Artificial Intelligence

Google

Machine learning

Artificial intelligence

Computer science

On January 1, 2025, which frontier AI lab will have a publicly available model with the highest score on the MMLU benchmark?

What will be the best score on the GAIA benchmark before 2025?

When will a language model be developed that, when tested, yields approximately human-level output?

2024-06-05

Comments

? comments

Authors:

Matthew_Barnett

Opened:

Closes:

Resolves:

AI Technical Benchmarks

Artificial Intelligence

Google

Machine learning

Artificial intelligence

Computer science

Researchers question AI’s ‘reasoning’ ability as models stumble on math problems with trivial changes

TechCrunch•Oct 12, 2024

The Techno-Agora: How LLMs Redefine Group Collaboration

Psychology Today•Oct 13, 2024

Human scientists are still better than AI ones – for now

New Scientist•Oct 16, 2024

Learn more about Metaculus NewsMatch

On January 1, 2025, which frontier AI lab will have a publicly available model with the highest score on the MMLU benchmark?

What will be the best score on the GAIA benchmark before 2025?

When will a language model be developed that, when tested, yields approximately human-level output?

2024-06-05