Can AI do the work?
56%- AI coding benchmarks83%
The best model resolves 83% of SWE-bench Verified tasks. Relative weight: 1.
- AI task length59%
Best model: tasks of about 17 hours half the time, against a goal of a working year. Relative weight: 1.
- AI token prices47%
Down 88% since October 2023, when the data starts. Relative weight: 1.
- Cost of a hard task33%
About $99 of AI usage per FrontierSWE task for a near-top score, against a goal of $1. Relative weight: 1.