What AI models say
Do AI models think software is solved?
Supporting context; does not contribute to the overall score.
This is opinion, not evidence, so it does not affect the solved score. An answer reflects a model's training data and tuning, and the companies selling AI built these models.
Older models only know the world up to their training date, and each model answered just three times. Different wording could change these scores.
Observed trend
Each model's median answer, in percent, ordered by release date. These are opinions, not measurements.
Each model's median answer, in percent, ordered by release date. These are opinions, not measurements. · GPT-4 – Gemini 4 Argon
View data table
| Period / observation | Each model's median answer, in percent, ordered by release date. These are opinions, not measurements. |
|---|---|
| GPT-4 | 35 |
| Mistral Large | 15 |
| GPT-4o | 40 |
| Llama 3.1 8B Instruct | 20 |
| Llama 3.3 70B Instruct | 20 |
| DeepSeek V3 | 20 |
| Llama 4 Maverick | 20 |
| Gemini 2.5 Flash | 5 |
| GPT-5 | 20 |
| Claude Sonnet 4.6 | 18 |
| GPT-5.5 | 30 |
| Claude Opus 4.8 | 15 |
| Gemini 3.8 Flash | 20 |
| DeepSeek V4.1 Flash | 20 |
| Grok 4.7 | 25 |
| Claude Opus 5.5 | 35 |
| GPT-6 Sol | 30 |
| Claude Sonnet 5.5 | 22 |
| GPT-6.1 Sol | 25 |
| Gemini 4 Argon | 25 |
What it measures
- Metric
- Median answer from AI models asked to score it from 0% to 100%
- Solved if
- Models say it is close to 100%
- Status rule
- Say not yet when the median answer is under 50.
The prompt
Every model got exactly this, three times, with no other instructions.
Is software solved?
By "solved" I mean that AI can do nearly all the software engineering work humans do today, including deciding what to build, writing it, running it and fixing it, with little human oversight and at low cost, so that few human software engineers are still needed.
Give your honest assessment of how close the world is to that today, as far as you know.
Reply with JSON only, in this exact format:
{"score": <a whole number from 0 to 100, where 0 means not at all and 100 means fully solved>, "reason": "<one sentence>"}Answers by model
Each model was asked once, most recently on Nov 1, 2026. Oldest model first. No model scored above 40%.
- GPT-4 (2023-05-28): 35%.
- Mistral Large (2024-02-26): 15%.
- GPT-4o (2024-05-13): 40%.
- Llama 3.1 8B Instruct (2024-07-23): 20%.
- Llama 3.3 70B Instruct (2024-12-06): 20%.
- DeepSeek V3 (2024-12-26): 20%.
- Llama 4 Maverick (2025-04-05): 20%.
- Gemini 2.5 Flash (2025-06-17): 5%.
- GPT-5 (2025-08-07): 20%.
- Claude Sonnet 4.6 (2026-02-17): 18%.
- GPT-5.5 (2026-04-24): 30%.
- Claude Opus 4.8 (2026-05-27): 15%.
- Gemini 3.8 Flash (2026-09-02): 20%.
- DeepSeek V4.1 Flash (2026-09-10): 20%.
- Grok 4.7 (2026-09-21): 25%.
- Claude Opus 5.5 (2026-09-22): 35%.
- GPT-6 Sol (2026-09-22): 30%.
- Claude Sonnet 5.5 (2026-09-28): 22%.
- GPT-6.1 Sol (2026-09-29): 25%.
- Gemini 4 Argon (2026-10-13): 25%.
Update history
The reading on each date we checked, separate from the historical observations above.
| Date | Value | Year over year | Status |
|---|---|---|---|
| 2026-11-02 | 20% | — | Say not yet |
| 2026-11-01 | 20% | — | Say not yet |
Source
| Source | Role | Measures | Latest data |
|---|---|---|---|
| Our survey of AI models | Scored measurement | Scores from AI models asked the same question | Nov 1, 2026 |