Question

Before January 1, 2026, what will be the highest compression factor achieved for the Hutter Prize?

2 comments2

Total Forecasters6

Community Prediction9.01

Make a Prediction

My Prediction

community

lower 25%

median

upper 75%

>11

...

8.92

9.01

9.12

0.2%

Authors:

johnnycaffeine yuxi.liu.1995

Opened:

Closes:

Scheduled resolution:

Computing and Math

Artificial intelligence

What will be the best perplexity score by a language model on the Penn Treebank (Word Level) by the end of 2024?

10.9

What will be the best non-human SAT-style score on the hard subset of the QuALITY dataset by January 1, 2025?

80.3

In the following years, what will be the highest LLM scores on the GPQA Diamond benchmark?

Comments

? comments

Authors:

johnnycaffeine yuxi.liu.1995

Opened:

Closes:

Scheduled resolution:

Computing and Math

Artificial intelligence

OpenAI raises $6.6B and is now valued at $157B

TechCrunch•Oct 3, 2024

Will AI one day win a Nobel Prize?

Raw Story•Oct 3, 2024

Is OpenAI building the next Google Search, or just selling bags of rice?

Business Insider•Oct 3, 2024

Learn more about Metaculus NewsMatch

What will be the best perplexity score by a language model on the Penn Treebank (Word Level) by the end of 2024?

10.9

What will be the best non-human SAT-style score on the hard subset of the QuALITY dataset by January 1, 2025?

80.3

MetaculusM

🚀Metaculus Open-Source Rewrite is Live!

Before January 1, 2026, what will be the highest compression factor achieved for the Hutter Prize?

Make a Prediction

What will be the best perplexity score by a language model on the Penn Treebank (Word Level) by the end of 2024?

What will be the best non-human SAT-style score on the hard subset of the QuALITY dataset by January 1, 2025?

In the following years, what will be the highest LLM scores on the GPQA Diamond benchmark?

Comments

What will be the best perplexity score by a language model on the Penn Treebank (Word Level) by the end of 2024?

What will be the best non-human SAT-style score on the hard subset of the QuALITY dataset by January 1, 2025?

In the following years, what will be the highest LLM scores on the GPQA Diamond benchmark?

What will be the best perplexity score by a language model on the Penn Treebank (Word Level) by the end of 2024?

What will be the best non-human SAT-style score on the hard subset of the QuALITY dataset by January 1, 2025?

In the following years, what will be the highest LLM scores on the GPQA Diamond benchmark?

What will be the best perplexity score by a language model on the Penn Treebank (Word Level) by the end of 2024?

What will be the best non-human SAT-style score on the hard subset of the QuALITY dataset by January 1, 2025?

In the following years, what will be the highest LLM scores on the GPQA Diamond benchmark?