Question

Which language modelling benchmark will be most popular in the calendar year 2022?

Resolved :Ambiguous

3 comments3

Total Forecasters30

Community Prediction2.99

Make a Prediction

My Prediction

community

lower 25%

median

upper 75%

...

2.168

2.988

3.054

What was the final result?Ambiguous

Authors:

Tamay

Opened:

Closes:

Resolves:

Computing and Math

Artificial Intelligence

What will be the state-of-the-art language modelling performance (in perplexity) on WikiText-103 by the following dates?

11.5

What will be the best perplexity score by a language model on the Penn Treebank (Word Level) by the end of 2024?

10.9

On January 1, 2025, which frontier AI lab will have a publicly available model with the highest score on the MMLU benchmark?

Comments

? comments

Authors:

Tamay

Opened:

Closes:

Resolves:

Computing and Math

Artificial Intelligence

What are AI ‘world models,’ and why do they matter?

TechCrunch•Oct 28, 2024

A mysterious new image generation model has appeared

TechCrunch•Oct 29, 2024

GitHub Copilot will support models from Anthropic, Google, and OpenAI

Verge•Oct 30, 2024

Learn more about Metaculus NewsMatch

What will be the state-of-the-art language modelling performance (in perplexity) on WikiText-103 by the following dates?

11.5

What will be the best perplexity score by a language model on the Penn Treebank (Word Level) by the end of 2024?

10.9