Question

When will a Chinese entity develop a model surpassing GPT-4's few-shot performance on MMLU?

18 comments18

Total Forecasters62

Community Prediction2024-12-10 (2024-09-28 - 2025-04-14)

Make a Prediction

weight:

My Prediction

community

lower 25%

median

upper 75%

>2029-06-13

...

2024-09-28 22:57

2024-12-10 09:41

2025-04-14 13:36

1.3%

Learn more about scores here.

Authors:

Lawrence

Opened:

Closes:

Scheduled resolution:

Business of AI

AI in China

Computing and Math

Artificial Intelligence

China

List of companies of China

The limits of intelligence — Why AI advancement could be slowing down

CNBC•Dec 13, 2024

In 2025 Gen AI Will Slow (Sundar Pichai) Or Sprint (Sam Altman)

Forbes•Dec 16, 2024

OpenAI Under Fire: Leadership Shake-ups and Rising Competition

•Dec 17, 2024

Learn more about Metaculus NewsMatch

On January 1, 2025, which frontier AI lab will have a publicly available model with the highest score on the MMLU benchmark?

How many months after a Western actor trains a 1e26 FLOP model (~5x GPT-4) will a Chinese actor do so?

11.7

When will an AI model trained with the following orders of magnitude more compute than GPT-4 be released?

Comments

? comments

Authors:

Lawrence

Opened:

Closes:

Scheduled resolution:

Business of AI

AI in China

Computing and Math

Artificial Intelligence

China

List of companies of China

The limits of intelligence — Why AI advancement could be slowing down

CNBC•Dec 13, 2024

In 2025 Gen AI Will Slow (Sundar Pichai) Or Sprint (Sam Altman)

Forbes•Dec 16, 2024

OpenAI Under Fire: Leadership Shake-ups and Rising Competition

•Dec 17, 2024

Learn more about Metaculus NewsMatch

On January 1, 2025, which frontier AI lab will have a publicly available model with the highest score on the MMLU benchmark?

How many months after a Western actor trains a 1e26 FLOP model (~5x GPT-4) will a Chinese actor do so?

11.7

Take Our 🏆Quarterly Cup🏆 Survey

MetaculusM

When will a Chinese entity develop a model surpassing GPT-4's few-shot performance on MMLU?

Make a Prediction

On January 1, 2025, which frontier AI lab will have a publicly available model with the highest score on the MMLU benchmark?

How many months after a Western actor trains a 1e26 FLOP model (~5x GPT-4) will a Chinese actor do so?

When will an AI model trained with the following orders of magnitude more compute than GPT-4 be released?

Comments

On January 1, 2025, which frontier AI lab will have a publicly available model with the highest score on the MMLU benchmark?

How many months after a Western actor trains a 1e26 FLOP model (~5x GPT-4) will a Chinese actor do so?

When will an AI model trained with the following orders of magnitude more compute than GPT-4 be released?

On January 1, 2025, which frontier AI lab will have a publicly available model with the highest score on the MMLU benchmark?

How many months after a Western actor trains a 1e26 FLOP model (~5x GPT-4) will a Chinese actor do so?

When will an AI model trained with the following orders of magnitude more compute than GPT-4 be released?

On January 1, 2025, which frontier AI lab will have a publicly available model with the highest score on the MMLU benchmark?

How many months after a Western actor trains a 1e26 FLOP model (~5x GPT-4) will a Chinese actor do so?

When will an AI model trained with the following orders of magnitude more compute than GPT-4 be released?