Question

What will the state-of-the-art language modelling performance on One Billion Word be on 2023-02-14, in perplexity, amongst models not trained on extra data?

Resolved :20.321

7 comments7

Total Forecasters36

Community Prediction21 (19.9 - 21.5)

Make a Prediction

PDF

CDF

My Prediction

community

<13

lower 25%

median

upper 75%

...

0.7%

19.93

21.5

What was the final result?20.3

Community Baseline Score

20.4

Community Peer Score

27.8

Authors:

MetaculusOutlooks

Opened:

Feb 13, 2021

Closes:

Apr 14, 2021

Resolves:

Feb 14, 2023

Forecasting AI Progress

AI Technical Benchmarks

Forecasting AI Progress: Deep Learning Round

Artificial Intelligence

2021-2025 Leaderboard

Machine learning

Linguistics

Data analysis

How AI and Human Brains Are Converging

Psychology Today•Jan 27, 2025

Viral AI company DeepSeek releases new image model family

TechCrunch•Jan 28, 2025

Why DeepSeek’s AI Model Just Became the Top-Rated App in the U.S.

Scientific American•Jan 28, 2025

Learn more about Metaculus NewsMatch

What will be the state-of-the-art language modelling performance (in perplexity) on WikiText-103 by the following dates?

11.5

How many billions of parameters will the largest machine learning model trained before 2030 have?

1.49M

When will a language model be developed that, when tested, yields approximately human-level output?

2024-06-05

Comments

? comments