Question

What will be the best score on the SWE-Bench (unassisted) benchmark before 2025?

15 comments15

Total Forecasters23

Community Prediction32.6

Make a Prediction

My Prediction

community

lower 25%

median

upper 75%

...

29.45

32.56

41.62

Authors:

romeodean

Opened:

Closes:

Scheduled resolution:

AI Technical Benchmarks

AI Benchmarks

Artificial Intelligence

Performance analysis

Software engineering

Comments

? comments

Authors:

romeodean

Opened:

Closes:

Scheduled resolution:

AI Technical Benchmarks

AI Benchmarks

Artificial Intelligence

Performance analysis

Software engineering

OpenAI is reportedly struggling to improve its next big AI model. It's a warning for the entire AI industry.

Business Insider•Nov 12, 2024

Anysphere acquires Supermaven to beef up Cursor

TechCrunch•Nov 13, 2024

OpenAI reportedly plans to launch an AI agent early next year

Verge•Nov 14, 2024

Learn more about Metaculus NewsMatch

What will be the best score on the WebArena benchmark before 2025?

64.9

When will models hit 90% on SWE-Bench (Verified Version)

2026-12-12

What will be the best score on the InterCode (Bash) benchmark before 2025?

55.6