AI benchmarks and leaderboards are different things
Learn the difference between a benchmark, an evaluation run and a leaderboard, and use each to answer the right question.
LLM Stats
Arena-Hard v2
Arena-Hard-Auto v2 is a challenging benchmark consisting of 500 carefully curated prompts sourced from Chatbot Arena and WildChat-1M, designed to evaluate large language models on real-world user queries. The benchmark covers diverse domains including open-ended software engineering problems, mathematics, creative writing, and technical problem-solving. It uses LLM-as-a-Judge for automatic evaluation, achieving 98.6% correlation with human preference rankings while providing 3x higher separation of model performances compared to MT-Bench. The benchmark emphasizes prompt specificity, complexity, and domain knowledge to better distinguish between model capabilities.
Evaluation context and evidence
- Source
- LLM Stats
- Publisher
- Not recorded
- Benchmark release
- Not recorded
- Collected
- 2026-08-17
- Modality
- text
These are source-reported observations, not independent reruns by this site. Check the task version, evaluation protocol, unit and reporting date before comparing models. Results from different setups are not a universal ranking.
Original sources
Reported results 16
Shown in source observation order, not ranked. Dates retain the precision provided by the source.
| Model | Value | Unit | Protocol | Reported date and basis | Source |
|---|---|---|---|---|---|
| MiMo-V2-Flash | 0.862 | Not recorded | Not recorded | 2025-12-16 · model_announcement | LLM Stats |
| Qwen3-Next-80B-A3B-Instruct | 0.827 | Not recorded | Not recorded | 2025-09-10 · model_announcement | LLM Stats |
| Qwen3-235B-A22B-Thinking-2507 | 0.797 | Not recorded | Not recorded | 2025-07-25 · model_announcement | LLM Stats |
| Qwen3-235B-A22B-Instruct-2507 | 0.792 | Not recorded | Not recorded | 2025-07-22 · model_announcement | LLM Stats |
| Qwen3 VL 235B A22B Instruct | 0.774 | Not recorded | Not recorded | 2025-09-22 · model_announcement | LLM Stats |
| Nemotron 3 Super (120B A12B) | 0.7388 | Not recorded | Not recorded | 2026-03-11 · model_announcement | LLM Stats |
| Sarvam-105B | 0.71 | Not recorded | Not recorded | 2026-03-06 · model_announcement | LLM Stats |
| Nemotron 3 Nano (30B A3B) | 0.677 | Not recorded | Not recorded | 2025-12-15 · model_announcement | LLM Stats |
| Qwen3 VL 32B Instruct | 0.647 | Not recorded | Not recorded | 2025-09-22 · model_announcement | LLM Stats |
| Qwen3-Next-80B-A3B-Thinking | 0.623 | Not recorded | Not recorded | 2025-09-10 · model_announcement | LLM Stats |
| Qwen3 VL 32B Thinking | 0.605 | Not recorded | Not recorded | 2025-09-22 · model_announcement | LLM Stats |
| Qwen3 VL 30B A3B Instruct | 0.585 | Not recorded | Not recorded | 2025-09-22 · model_announcement | LLM Stats |
| Qwen3 VL 30B A3B Thinking | 0.567 | Not recorded | Not recorded | 2025-09-22 · model_announcement | LLM Stats |
| Qwen3 VL 8B Thinking | 0.511 | Not recorded | Not recorded | 2025-09-22 · model_announcement | LLM Stats |
| Sarvam-30B | 0.49 | Not recorded | Not recorded | 2026-03-06 · model_announcement | LLM Stats |
| Qwen3 VL 4B Thinking | 0.368 | Not recorded | Not recorded | 2025-09-22 · model_announcement | LLM Stats |