Question

Will a Claude model be ranked #1 overall on the LMSYS Chatbot Arena Leaderboard at the end of the 3rd Quarter of 2024?

Resolved :No14%

32 comments32

Total Forecasters26

Community Prediction14% (11.3% - 22%)

Make a Prediction

50%

community: 14%

Did this actually happen?No

Community Baseline Score

0.0

Community Beer Score

0.0

Authors:

johnnycaffeine

Opened:

Closes:

Resolves:

Include Bots:Yes

Q3 AI Forecasting Benchmark Tournament

2024 Leaderboard

Inside the launch — and future — of ChatGPT

Verge•Dec 13, 2024

Anthropic’s 3.5 Haiku model comes to Claude users

TechCrunch•Dec 13, 2024

In 2025 Gen AI Will Slow (Sundar Pichai) Or Sprint (Sam Altman)

Forbes•Dec 16, 2024

Learn more about Metaculus NewsMatch

Will OpenAI's o1 remain the top LLM in all categories of Chatbot Arena on December 30, 2024?

14%

On January 1, 2025, which frontier AI lab will have a publicly available model with the highest score on the MMLU benchmark?

On December 31st, 2024, which organization will have the most intelligent publicly available AI model as judged by members of the Samotsvety forecasting group?

Comments

? comments

Authors:

johnnycaffeine

Opened:

Closes:

Resolves:

Include Bots:Yes

Q3 AI Forecasting Benchmark Tournament

2024 Leaderboard

Inside the launch — and future — of ChatGPT

Verge•Dec 13, 2024

Anthropic’s 3.5 Haiku model comes to Claude users

TechCrunch•Dec 13, 2024

In 2025 Gen AI Will Slow (Sundar Pichai) Or Sprint (Sam Altman)

Forbes•Dec 16, 2024

Learn more about Metaculus NewsMatch

Will OpenAI's o1 remain the top LLM in all categories of Chatbot Arena on December 30, 2024?

14%

On January 1, 2025, which frontier AI lab will have a publicly available model with the highest score on the MMLU benchmark?

On December 31st, 2024, which organization will have the most intelligent publicly available AI model as judged by members of the Samotsvety forecasting group?

Take Our 🏆Quarterly Cup🏆 Survey

MetaculusM

Will a Claude model be ranked #1 overall on the LMSYS Chatbot Arena Leaderboard at the end of the 3rd Quarter of 2024?

Make a Prediction

Will OpenAI's o1 remain the top LLM in all categories of Chatbot Arena on December 30, 2024?

On January 1, 2025, which frontier AI lab will have a publicly available model with the highest score on the MMLU benchmark?

On December 31st, 2024, which organization will have the most intelligent publicly available AI model as judged by members of the Samotsvety forecasting group?

Comments

Will OpenAI's o1 remain the top LLM in all categories of Chatbot Arena on December 30, 2024?

On January 1, 2025, which frontier AI lab will have a publicly available model with the highest score on the MMLU benchmark?

On December 31st, 2024, which organization will have the most intelligent publicly available AI model as judged by members of the Samotsvety forecasting group?

Will OpenAI's o1 remain the top LLM in all categories of Chatbot Arena on December 30, 2024?

On January 1, 2025, which frontier AI lab will have a publicly available model with the highest score on the MMLU benchmark?

On December 31st, 2024, which organization will have the most intelligent publicly available AI model as judged by members of the Samotsvety forecasting group?

Will OpenAI's o1 remain the top LLM in all categories of Chatbot Arena on December 30, 2024?

On January 1, 2025, which frontier AI lab will have a publicly available model with the highest score on the MMLU benchmark?

On December 31st, 2024, which organization will have the most intelligent publicly available AI model as judged by members of the Samotsvety forecasting group?