Data source
Our survey of AI models
We asked AI models from several labs and generations one fixed prompt, three times each, through OpenRouter, with no other instructions. New models are added after release, and earlier answers are kept. We publish the prompt and the scores.
Provides measurements used by the linked signal.
Our own survey of AI models, run through OpenRouter.
Visit the sourceUsed for: What AI models sayLicense: Our own run
Median answer20%
Models20
What the data shows
Computed from every record, as of Nov 1, 2026.
- Last asked on Nov 1, 2026, 20 AI models gave a median score of 20%.
Answers by model
| Model | Released | Median answer |
|---|---|---|
| GPT-4 | 2023-05-28 | 35% |
| Mistral Large | 2024-02-26 | 15% |
| GPT-4o | 2024-05-13 | 40% |
| Llama 3.1 8B Instruct | 2024-07-23 | 20% |
| Llama 3.3 70B Instruct | 2024-12-06 | 20% |
| DeepSeek V3 | 2024-12-26 | 20% |
| Llama 4 Maverick | 2025-04-05 | 20% |
| Gemini 2.5 Flash | 2025-06-17 | 5% |
| GPT-5 | 2025-08-07 | 20% |
| Claude Sonnet 4.6 | 2026-02-17 | 18% |
| GPT-5.5 | 2026-04-24 | 30% |
| Claude Opus 4.8 | 2026-05-27 | 15% |
| Gemini 3.8 Flash | 2026-09-02 | 20% |
| DeepSeek V4.1 Flash | 2026-09-10 | 20% |
| Grok 4.7 | 2026-09-21 | 25% |
| Claude Opus 5.5 | 2026-09-22 | 35% |
| GPT-6 Sol | 2026-09-22 | 30% |
| Claude Sonnet 5.5 | 2026-09-28 | 22% |
| GPT-6.1 Sol | 2026-09-29 | 25% |
| Gemini 4 Argon | 2026-10-13 | 25% |
Limitations
- Each model is asked once, and it can give a different answer on another day.
- Models only appear if OpenRouter offered them when we asked, so the list is not complete.