MetaculusM

Questions
Tournaments

Leaderboards
News
create

AI Benchmarking

Topics

categories

See all categories

About
API

FAQ
forecasting resources
For Journalists

Careers

Guidelines Privacy Policy Terms of Use

Will OpenAI's o1 remain the top LLM in all categories of Chatbot Arena on December 30, 2024?

135 forecasters

14

🏆 Quarterly Cup 🏆

When will the first weakly general AI system be devised, tested, and publicly announced?

1523 forecasters

201

482 comments482

AI Progress Essay Contest

Will an AI be able to work as a competent cook in an arbitrary kitchen before 2030?

306 forecasters

33

AI Demonstrations

Will the winning bot in any Quarterly AI Benchmarking tournament beat the human Pro aggregate before Q3 of 2025?

8

When will models hit 90% on SWE-Bench (Verified Version)

3

David Mathers' Community

When will a Chinese entity develop a model surpassing GPT-4's few-shot performance on MMLU?

13

On January 1, 2025, which frontier AI lab will have a publicly available model with the highest score on the MMLU benchmark?

Google DeepMind

9

Understanding AI With Timothy B. Lee

Before 2025, will laws be in place requiring that AI systems that emulate humans must reveal to people that they are AI?

Resolved :Annulled

213 forecasters

28

Regulation of AI

What will be state-of-the-art accuracy on the Massive Multitask dataset on the following dates?

June 30, 2025:94.7

12

AI Technical Benchmarks

How many of the 10 most important advancements in machine learning or artificial intelligence of 2025-2030 will have been discovered by an AI system?

1

Threshold 2030 (Day One)