AI benchmarks and leaderboards are different things
Learn the difference between a benchmark, an evaluation run and a leaderboard, and use each to answer the right question.
LLM Stats
Aider-Polyglot
A coding benchmark that evaluates LLMs on 225 challenging Exercism programming exercises across C++, Go, Java, JavaScript, Python, and Rust. Models receive two attempts to solve each problem, with test error feedback provided after the first attempt if it fails. The benchmark measures both initial problem-solving ability and capacity to edit code based on error feedback, providing an end-to-end evaluation of code generation and editing capabilities across multiple programming languages.
Evaluation context and evidence
- Source
- LLM Stats
- Publisher
- Not recorded
- Benchmark release
- Not recorded
- Collected
- 2026-08-17
- Modality
- text
These are source-reported observations, not independent reruns by this site. Check the task version, evaluation protocol, unit and reporting date before comparing models. Results from different setups are not a universal ranking.
Original sources
Reported results 22
Shown in source observation order, not ranked. Dates retain the precision provided by the source.
| Model | Value | Unit | Protocol | Reported date and basis | Source |
|---|---|---|---|---|---|
| GPT-5 | 0.88 | Not recorded | Not recorded | 2025-08-07 · model_announcement | LLM Stats |
| Gemini 2.5 Pro Preview 06-05 | 0.822 | Not recorded | Not recorded | 2025-06-05 · model_announcement | LLM Stats |
| o3 | 0.813 | Not recorded | Not recorded | 2025-04-16 · model_announcement | LLM Stats |
| Gemini 2.5 Pro | 0.765 | Not recorded | Not recorded | 2025-05-20 · model_announcement | LLM Stats |
| DeepSeek-V3.2-Exp | 0.745 | Not recorded | Not recorded | 2025-09-29 · model_announcement | LLM Stats |
| DeepSeek-R1-0528 | 0.716 | Not recorded | Not recorded | 2025-05-28 · model_announcement | LLM Stats |
| o4-mini | 0.689 | Not recorded | Not recorded | 2025-04-16 · model_announcement | LLM Stats |
| DeepSeek-V3.1 | 0.684 | Not recorded | Not recorded | 2025-01-10 · model_announcement | LLM Stats |
| o3-mini | 0.667 | Not recorded | Not recorded | 2025-01-30 · model_announcement | LLM Stats |
| Gemini 2.5 Flash | 0.619 | Not recorded | Not recorded | 2025-05-20 · model_announcement | LLM Stats |
| Qwen3-Coder 480B A35B Instruct | 0.618 | Not recorded | Not recorded | 2025-01-31 · model_announcement | LLM Stats |
| Kimi K2-Instruct-0905 | 0.6 | Not recorded | Not recorded | 2025-09-05 · model_announcement | LLM Stats |
| Kimi K2 Instruct | 0.6 | Not recorded | Not recorded | 2025-07-11 · model_announcement | LLM Stats |
| Qwen3-235B-A22B-Instruct-2507 | 0.573 | Not recorded | Not recorded | 2025-07-22 · model_announcement | LLM Stats |
| GPT-4.1 | 0.516 | Not recorded | Not recorded | 2025-04-14 · model_announcement | LLM Stats |
| Qwen3-Next-80B-A3B-Instruct | 0.498 | Not recorded | Not recorded | 2025-09-10 · model_announcement | LLM Stats |
| DeepSeek-V3 | 0.496 | Not recorded | Not recorded | 2024-12-25 · model_announcement | LLM Stats |
| Magistral Medium | 0.471 | Not recorded | Not recorded | 2025-06-10 · model_announcement | LLM Stats |
| GPT-4.1 mini | 0.347 | Not recorded | Not recorded | 2025-04-14 · model_announcement | LLM Stats |
| GPT-4o | 0.307 | Not recorded | Not recorded | 2024-08-06 · model_announcement | LLM Stats |
| Gemini 2.5 Flash-Lite | 0.267 | Not recorded | Not recorded | 2025-06-17 · model_announcement | LLM Stats |
| GPT-4.1 nano | 0.098 | Not recorded | Not recorded | 2025-04-14 · model_announcement | LLM Stats |