MetaculusM

Questions
Tournaments

Leaderboards
News
create

AI Benchmarking

Topics

categories

See all categories

About
API

FAQ
forecasting resources
For Journalists

Careers

Guidelines Privacy Policy Terms of Use

If GPT-4 is announced before 2025, how many parameters will it have (in billions of parameters)?

10

When will the first weakly general AI system be devised, tested, and publicly announced?

1523 forecasters

201

482 comments482

AI Progress Essay Contest

Will a member of the United States Congress introduce legislation limiting the use of LLMs in 2024?

499 forecasters

9

ACX 2024 Prediction Contest

On January 1, 2025, which frontier AI lab will have a publicly available model with the highest score on the MMLU benchmark?

Google DeepMind

9

Understanding AI With Timothy B. Lee

What will be state-of-the-art accuracy on the Massive Multitask dataset on the following dates?

June 30, 2025:94.7

12

AI Technical Benchmarks

Will a book written by a language model make the NY Times Best Seller list before 2030?

266 forecasters

47

AI Demonstrations

Will there be Human-machine intelligence parity before 2040?

1459 forecasters

221

146 comments146

Will the listed AI companies/labs pause any of their models before January 1st, 2025 because they detect dangerous capabilities?

Google DeepMind

12

Will the listed AI companies/labs create a board risk committee before January 1st, 2025?

Google DeepMind

3

What will be the best perplexity score by a language model on the Penn Treebank (Word Level) by the end of 2024?

5

AI Technical Benchmarks