AI benchmarks and leaderboards are different things
Learn the difference between a benchmark, an evaluation run and a leaderboard, and use each to answer the right question.
LLM Stats
Multi-Challenge
MultiChallenge is a realistic multi-turn conversation evaluation benchmark that challenges frontier LLMs across four key categories: instruction retention (maintaining instructions throughout conversations), inference memory (recalling and connecting details from previous turns), reliable versioned editing (adapting to evolving instructions during collaborative editing), and self-coherence (avoiding contradictions in responses). The benchmark evaluates models on sustained, contextually complex dialogues across diverse topics including travel planning, technical documentation, and professional communication.
Evaluation context and evidence
- Source
- LLM Stats
- Publisher
- Not recorded
- Benchmark release
- Not recorded
- Collected
- 2026-08-17
- Modality
- text
These are source-reported observations, not independent reruns by this site. Check the task version, evaluation protocol, unit and reporting date before comparing models. Results from different setups are not a universal ranking.
Original sources
Reported results 29
Shown in source observation order, not ranked. Dates retain the precision provided by the source.
| Model | Value | Unit | Protocol | Reported date and basis | Source |
|---|---|---|---|---|---|
| Nova 2 Pro | 0.777 | Not recorded | Not recorded | 2025-12-02 · model_announcement | LLM Stats |
| Nova 2 Lite | 0.766 | Not recorded | Not recorded | 2025-12-02 · model_announcement | LLM Stats |
| Nova 2 Omni | 0.755 | Not recorded | Not recorded | 2025-12-02 · model_announcement | LLM Stats |
| GPT-5 | 0.696 | Not recorded | Not recorded | 2025-08-07 · model_announcement | LLM Stats |
| Qwen3.5-397B-A17B | 0.676 | Not recorded | Not recorded | 2026-02-16 · model_announcement | LLM Stats |
| Nemotron 3 Ultra (550B A55B) | 0.638 | Not recorded | Not recorded | 2026-06-04 · model_announcement | LLM Stats |
| Step3-VL-10B | 0.626 | Not recorded | Not recorded | 2026-01-15 · model_announcement | LLM Stats |
| Qwen3.5-122B-A10B | 0.615 | Not recorded | Not recorded | 2026-02-24 · model_announcement | LLM Stats |
| Qwen3.5-27B | 0.608 | Not recorded | Not recorded | 2026-02-24 · model_announcement | LLM Stats |
| o3 | 0.604 | Not recorded | Not recorded | 2025-04-16 · model_announcement | LLM Stats |
| Qwen3.5-35B-A3B | 0.6 | Not recorded | Not recorded | 2026-02-24 · model_announcement | LLM Stats |
| Nemotron 3 Super (120B A12B) | 0.5523 | Not recorded | Not recorded | 2026-03-11 · model_announcement | LLM Stats |
| Qwen3.5-9B | 0.545 | Not recorded | Not recorded | 2026-03-02 · model_announcement | LLM Stats |
| Kimi K2-Instruct-0905 | 0.541 | Not recorded | Not recorded | 2025-09-05 · model_announcement | LLM Stats |
| Kimi K2 Instruct | 0.541 | Not recorded | Not recorded | 2025-07-11 · model_announcement | LLM Stats |
| MAI-Thinking-1 | 0.53 | Not recorded | Not recorded | 2026-06-02 · model_announcement | LLM Stats |
| Qwen3.5-4B | 0.49 | Not recorded | Not recorded | 2026-03-02 · model_announcement | LLM Stats |
| MiniMax M1 80K | 0.447 | Not recorded | Not recorded | 2025-06-16 · model_announcement | LLM Stats |
| MiniMax M1 40K | 0.447 | Not recorded | Not recorded | 2025-06-16 · model_announcement | LLM Stats |
| GPT-4.5 | 0.438 | Not recorded | Not recorded | 2025-02-27 · model_announcement | LLM Stats |
| o4-mini | 0.43 | Not recorded | Not recorded | 2025-04-16 · model_announcement | LLM Stats |
| GPT-4o | 0.403 | Not recorded | Not recorded | 2024-08-06 · model_announcement | LLM Stats |
| o3-mini | 0.399 | Not recorded | Not recorded | 2025-01-30 · model_announcement | LLM Stats |
| Nemotron 3 Nano (30B A3B) | 0.385 | Not recorded | Not recorded | 2025-12-15 · model_announcement | LLM Stats |
| GPT-4.1 | 0.383 | Not recorded | Not recorded | 2025-04-14 · model_announcement | LLM Stats |
| GPT-4.1 mini | 0.358 | Not recorded | Not recorded | 2025-04-14 · model_announcement | LLM Stats |
| Qwen3.5-2B | 0.337 | Not recorded | Not recorded | 2026-03-02 · model_announcement | LLM Stats |
| Qwen3.5-0.8B | 0.189 | Not recorded | Not recorded | 2026-03-02 · model_announcement | LLM Stats |
| GPT-4.1 nano | 0.15 | Not recorded | Not recorded | 2025-04-14 · model_announcement | LLM Stats |