AI coding benchmarks
Can AI models do real engineering tasks?
Contributes to: Can AI do the work?. Signal score: 83 / 100. Relative weight within this question: 1.
A benchmark is not a job. These tasks are fixed in advance and scored automatically, so a high score does not show that a model can run a real software project.
Scores differ widely between benchmarks, and models can be tuned for well-known tests. New models are not always tested right away, so the latest result can lag.
Observed trend
Best SWE-bench Verified score so far, in percent, by model release month.
Best SWE-bench Verified score so far, in percent, by model release month. · 2024-11 – 2026-10
View data table
| Period / observation | Best SWE-bench Verified score so far, in percent, by model release month. |
|---|---|
| 2024-11 | 31 |
| 2024-12 | 31 |
| 2025-01 | 31 |
| 2025-02 | 61 |
| 2025-03 | 61 |
| 2025-04 | 62 |
| 2025-05 | 71 |
| 2025-06 | 71 |
| 2025-07 | 71 |
| 2025-08 | 74 |
| 2025-09 | 74 |
| 2025-10 | 74 |
| 2025-11 | 77 |
| 2025-12 | 77 |
| 2026-01 | 77 |
| 2026-02 | 79 |
| 2026-03 | 79 |
| 2026-04 | 83 |
| 2026-05 | 83 |
| 2026-06 | 83 |
| 2026-07 | 83 |
| 2026-08 | 83 |
| 2026-09 | 83 |
| 2026-10 | 83 |
What it measures
- Metric
- Best score so far on SWE-bench Verified, the share of real GitHub issues resolved
- Solved if
- Models resolve nearly every real engineering task
- Signal score
- 83 / 100. The best model resolves 83% of SWE-bench Verified tasks. How it is scored
- Status rule
- Improving when the best score is higher than a year before.
Other public benchmarks
The best score depends heavily on the benchmark, so no single number settles this.
- MirrorCode: 79%, from claude-opus-5-5.
- DeepSWE: 74%, from gpt-6-astra.
- FrontierSWE: 66%, from gpt-6-astra.
- FrontierCode: 55%, from claude-opus-5-5.
- METR estimates the longest task a model completes half the time at about 17 hours of human work, with a range of 8 to 55 hours.
Update history
The reading on each date we checked, separate from the historical observations above.
| Date | Value | Year over year | Status |
|---|---|---|---|
| 2026-11-02 | 83% | +12% | Improving |
| 2026-11-01 | 83% | +12% | Improving |
Sources
| Source | Role | Measures | Latest data |
|---|---|---|---|
| Epoch AI Benchmarking Hub | Scored measurement | Model scores on coding benchmarks | 2026-04 |
| METR Time Horizons | Supporting context | Length of task a model completes half the time | 2026-10 |