Group

What percentage of problems from Long Code Arena do you estimate the top-performing AI model or agent will be able to solve by December'25?

no comments0

Forecast Timeline

Continuous integration builds repair

Project-level code completion

Commit message generation

Library-based code generation

Bug localization

Module summarization

Make a Prediction

Long Code Arena benchmarks

median

PDF

Authors:

ilia.zaharov

Opened:

Feb 12, 2025

Closes:

Nov 30, 2025

Scheduled resolution:

Dec 14, 2025

JetBrains

Artificial Intelligence

Innovative Minecraft AI Benchmark: High Schooler’s Genius Website Challenges Generative AI Models

•Mar 23, 2025

High Schooler Builds Website for AI Minecraft Battles

•Mar 23, 2025

Summary by AI

•Mar 23, 2025

Learn more about Metaculus NewsMatch

For these benchmarks, what percentage of problems do you estimate the top-performing AI model or agent will be able to solve by December'25?

54.2

When will an AI be able to solve over 80% of competition-level coding problems on its first try?

2027

What will state-of-the-art top-1 accuracy on the APPS Benchmark introductory problems be from 2022 to 2025?

91.6

Comments

? comments

Authors:

ilia.zaharov

Opened:

Feb 12, 2025

Closes:

Nov 30, 2025

Scheduled resolution:

Dec 14, 2025

JetBrains

Artificial Intelligence

Innovative Minecraft AI Benchmark: High Schooler’s Genius Website Challenges Generative AI Models

•Mar 23, 2025

High Schooler Builds Website for AI Minecraft Battles

•Mar 23, 2025

Summary by AI

•Mar 23, 2025

Learn more about Metaculus NewsMatch

For these benchmarks, what percentage of problems do you estimate the top-performing AI model or agent will be able to solve by December'25?

54.2

When will an AI be able to solve over 80% of competition-level coding problems on its first try?

2027

What will state-of-the-art top-1 accuracy on the APPS Benchmark introductory problems be from 2022 to 2025?

91.6