Question

What will the state-of-the-art language modelling performance on One Billion Word be on 2022-01-14, in perplexity?

Resolved :20.321.4

5 comments5

Total Forecasters41

Community Prediction21.4 (20.8 - 21.6)

Make a Prediction

PDF

CDF

My Prediction

community

<14

lower 25%

median

upper 75%

...

0.5%

20.78

21.39

21.59

What was the final result?20.3

Community Baseline Score

12.7

Community Peer Score

36.5

Authors:

MetaculusOutlooks

Opened:

Jan 14, 2021

Closes:

Mar 14, 2021

Resolves:

Mar 22, 2022

Forecasting AI Progress

Forecasting AI Progress: Hill Climbing Round

Artificial Intelligence

2021 Leaderboard

Linguistics

Data analysis

DeepSeek just dropped an update AI model called Janus Pro. It says the image generator is better than OpenAI's DALL-E.

Business Insider•Jan 28, 2025

Viral AI company DeepSeek releases new image model family

TechCrunch•Jan 28, 2025

Why DeepSeek’s AI Model Just Became the Top-Rated App in the U.S.

Scientific American•Jan 28, 2025

Learn more about Metaculus NewsMatch

What will be the state-of-the-art language modelling performance (in perplexity) on WikiText-103 by the following dates?

11.5

When will a language model be developed that, when tested, yields approximately human-level output?

2024-06-05

How many billions of parameters will the largest machine learning model trained before 2030 have?

1.49M

Comments

? comments