Question

When will AI achieve competency on multi-choice questions across diverse fields of expertise (MMLU)?

27 comments27

Total Forecasters86

Community Prediction2025-07-27 (2024-10-07 - 2026-09-13)

Make a Prediction

weight:

My Prediction

community

lower 25%

median

upper 75%

>2039-12-31

...

2024-10-07 03:59

2025-07-27 05:22

2026-09-13 04:16

1.1%

Learn more about scores here.

Authors:

isinlor

Opened:

Closes:

Scheduled resolution:

AI Demonstrations

Computing and Math

Artificial Intelligence

Machine learning

Why AI must learn to admit ignorance and say 'I don't know'

New Scientist•Dec 13, 2024

Will 2025 Be a "Technology Wake-Up Call" for Clinicians?

Psychology Today•Dec 20, 2024

OpenAI Upgrades Its Smartest AI Model With Improved Reasoning Skills

Wired•Dec 21, 2024

Learn more about Metaculus NewsMatch

On January 1, 2025, which frontier AI lab will have a publicly available model with the highest score on the MMLU benchmark?

What will be state-of-the-art accuracy on the Massive Multitask dataset on the following dates?

94.6

When will an AI be able to solve over 80% of competition-level coding problems on its first try?

2027-03-18

Comments

? comments

Authors:

isinlor

Opened:

Closes:

Scheduled resolution:

AI Demonstrations

Computing and Math

Artificial Intelligence

Machine learning

Why AI must learn to admit ignorance and say 'I don't know'

New Scientist•Dec 13, 2024

Will 2025 Be a "Technology Wake-Up Call" for Clinicians?

Psychology Today•Dec 20, 2024

OpenAI Upgrades Its Smartest AI Model With Improved Reasoning Skills

Wired•Dec 21, 2024

Learn more about Metaculus NewsMatch

On January 1, 2025, which frontier AI lab will have a publicly available model with the highest score on the MMLU benchmark?

What will be state-of-the-art accuracy on the Massive Multitask dataset on the following dates?

94.6

When will an AI be able to solve over 80% of competition-level coding problems on its first try?

2027-03-18

Take Our 🏆Quarterly Cup🏆 Survey

MetaculusM

When will AI achieve competency on multi-choice questions across diverse fields of expertise (MMLU)?

Make a Prediction

On January 1, 2025, which frontier AI lab will have a publicly available model with the highest score on the MMLU benchmark?

What will be state-of-the-art accuracy on the Massive Multitask dataset on the following dates?

When will an AI be able to solve over 80% of competition-level coding problems on its first try?

Comments

On January 1, 2025, which frontier AI lab will have a publicly available model with the highest score on the MMLU benchmark?

What will be state-of-the-art accuracy on the Massive Multitask dataset on the following dates?

When will an AI be able to solve over 80% of competition-level coding problems on its first try?

On January 1, 2025, which frontier AI lab will have a publicly available model with the highest score on the MMLU benchmark?

What will be state-of-the-art accuracy on the Massive Multitask dataset on the following dates?

When will an AI be able to solve over 80% of competition-level coding problems on its first try?

On January 1, 2025, which frontier AI lab will have a publicly available model with the highest score on the MMLU benchmark?

What will be state-of-the-art accuracy on the Massive Multitask dataset on the following dates?

When will an AI be able to solve over 80% of competition-level coding problems on its first try?