Take Our 🏆Quarterly Cup🏆 Survey

We're considering changes to the Quarterly Cup and want to hear from you, take our short survey to share your thoughts and help shape future iterations of the Cup.

M

Question

What will be the best score on the SWE-Bench (unassisted) benchmark before 2025?

9
17
Total Forecasters23
Community Prediction31.3 (29.5 - 38.2)

Make a Prediction

weight:
My Prediction
community
lower 25%
median
upper 75%
...
...
...
29.53
31.29
38.18
Learn more about scores here.
Authors:
romeodean
Opened:
Closes:
Scheduled resolution:
AI Technical Benchmarks
AI Benchmarks
Artificial Intelligence
2024 Leaderboard
Performance analysis
Software engineering

Comments

? comments