AI benchmarks and leaderboards are different things
Learn the difference between a benchmark, an evaluation run and a leaderboard, and use each to answer the right question.
LLM Stats
MMMU-Pro
A more robust multi-discipline multimodal understanding benchmark that enhances MMMU through a three-step process: filtering text-only answerable questions, augmenting candidate options, and introducing vision-only input settings. Achieves significantly lower model performance (16.8-26.9%) compared to original MMMU, providing more rigorous evaluation that closely mimics real-world scenarios.
Evaluation context and evidence
- Source
- LLM Stats
- Publisher
- Not recorded
- Benchmark release
- 2024-09-04
- Collected
- 2026-08-17
- Modality
- multimodal
These are source-reported observations, not independent reruns by this site. Check the task version, evaluation protocol, unit and reporting date before comparing models. Results from different setups are not a universal ranking.
Original sources
Reported results 68
Shown in source observation order, not ranked. Dates retain the precision provided by the source.
| Model | Value | Unit | Protocol | Reported date and basis | Source |
|---|---|---|---|---|---|
| Gemini 3.5 Flash | 0.836 | Not recorded | Not recorded | 2026-05-19 · model_announcement | LLM Stats |
| GPT-5.5 | 0.832 | Not recorded | Not recorded | 2026-04-23 · model_announcement | LLM Stats |
| GPT-5.6 Sol | 0.83 | Not recorded | Not recorded | 2026-07-09 · model_announcement | LLM Stats |
| Seed 2.1 Pro | 0.827 | Not recorded | Not recorded | 2026-06-24 · model_announcement | LLM Stats |
| Qwen3.8 Max | 0.823 | Not recorded | Not recorded | 2026-08-02 · model_announcement | LLM Stats |
| Seed 2.1 Turbo | 0.822 | Not recorded | Not recorded | 2026-06-24 · model_announcement | LLM Stats |
| Kimi K3 | 0.816 | Not recorded | Not recorded | 2026-07-16 · model_announcement | LLM Stats |
| GPT-5.4 | 0.812 | Not recorded | Not recorded | 2026-03-05 · model_announcement | LLM Stats |
| Gemini 3 Flash | 0.812 | Not recorded | Not recorded | 2025-12-17 · model_announcement | LLM Stats |
| Gemini 3 Pro | 0.81 | Not recorded | Not recorded | 2025-11-18 · model_announcement | LLM Stats |
| GPT-5.6 Terra | 0.807 | Not recorded | Not recorded | 2026-07-09 · model_announcement | LLM Stats |
| Gemini 3.1 Pro | 0.805 | Not recorded | Not recorded | 2026-02-19 · model_announcement | LLM Stats |
| Muse Spark | 0.804 | Not recorded | Not recorded | 2026-04-08 · model_announcement | LLM Stats |
| Kimi K2.6 | 0.801 | Not recorded | Not recorded | 2026-04-20 · model_announcement | LLM Stats |
| GPT-5.2 | 0.795 | Not recorded | Not recorded | 2025-12-11 · model_announcement | LLM Stats |
| Qwen3.7-Plus | 0.79 | Not recorded | Not recorded | 2026-05-31 · model_announcement | LLM Stats |
| Qwen3.6 Plus | 0.788 | Not recorded | Not recorded | 2026-04-02 · model_announcement | LLM Stats |
| Kimi K2.5 | 0.785 | Not recorded | Not recorded | 2026-01-27 · model_announcement | LLM Stats |
| GPT-5.6 Luna | 0.784 | Not recorded | Not recorded | 2026-07-09 · model_announcement | LLM Stats |
| GPT-5 | 0.784 | Not recorded | Not recorded | 2025-08-07 · model_announcement | LLM Stats |
| MiniMax M3 | 0.781 | Not recorded | Not recorded | 2026-06-01 · model_announcement | LLM Stats |
| MiMo-V2.5 | 0.779 | Not recorded | Not recorded | 2026-04-22 · model_announcement | LLM Stats |
| Claude Opus 4.6 | 0.773 | Not recorded | Not recorded | 2026-02-05 · model_announcement | LLM Stats |
| Gemma 4 31B | 0.769 | Not recorded | Not recorded | 2026-04-02 · model_announcement | LLM Stats |
| Qwen3.5-122B-A10B | 0.769 | Not recorded | Not recorded | 2026-02-24 · model_announcement | LLM Stats |
| Gemini 3.1 Flash-Lite | 0.768 | Not recorded | Not recorded | 2026-03-03 · model_announcement | LLM Stats |
| GPT-5.4 mini | 0.766 | Not recorded | Not recorded | 2026-03-17 · model_announcement | LLM Stats |
| o3 | 0.764 | Not recorded | Not recorded | 2025-04-16 · model_announcement | LLM Stats |
| GPT-5.5 Instant | 0.76 | Not recorded | Not recorded | 2026-05-05 · model_announcement | LLM Stats |
| Qwen3.6-27B | 0.758 | Not recorded | Not recorded | 2026-04-21 · model_announcement | LLM Stats |
| Claude Sonnet 4.6 | 0.756 | Not recorded | Not recorded | 2026-02-17 · model_announcement | LLM Stats |
| Qwen3.6-35B-A3B | 0.753 | Not recorded | Not recorded | 2026-04-16 · model_announcement | LLM Stats |
| Qwen3.5-35B-A3B | 0.751 | Not recorded | Not recorded | 2026-02-24 · model_announcement | LLM Stats |
| Qwen3.5-27B | 0.75 | Not recorded | Not recorded | 2026-02-24 · model_announcement | LLM Stats |
| Muse Glimmer-30B | 0.74 | Not recorded | Not recorded | 2026-08-10 · model_announcement | LLM Stats |
| Inkling-Small | 0.74 | Not recorded | Not recorded | 2026-07-30 · model_announcement | LLM Stats |
| Gemma 4 26B-A4B | 0.738 | Not recorded | Not recorded | 2026-04-02 · model_announcement | LLM Stats |
| Qwen3 VL 235B A22B Thinking | 0.693 | Not recorded | Not recorded | 2025-09-22 · model_announcement | LLM Stats |
| Gemma 4 12B | 0.691 | Not recorded | Not recorded | 2026-05-23 · model_announcement | LLM Stats |
| Qwen3 VL 32B Thinking | 0.681 | Not recorded | Not recorded | 2025-09-22 · model_announcement | LLM Stats |
| Qwen3 VL 235B A22B Instruct | 0.681 | Not recorded | Not recorded | 2025-09-22 · model_announcement | LLM Stats |
| GPT-5.4 nano | 0.661 | Not recorded | Not recorded | 2026-03-17 · model_announcement | LLM Stats |
| Qwen3 VL 32B Instruct | 0.653 | Not recorded | Not recorded | 2025-09-22 · model_announcement | LLM Stats |
| Nova 2 Pro | 0.635 | Not recorded | Not recorded | 2025-12-02 · model_announcement | LLM Stats |
| Command A+ | 0.63 | Not recorded | Not recorded | 2026-05-20 · model_announcement | LLM Stats |
| Qwen3 VL 30B A3B Thinking | 0.63 | Not recorded | Not recorded | 2025-09-22 · model_announcement | LLM Stats |
| Nova 2 Lite | 0.618 | Not recorded | Not recorded | 2025-12-02 · model_announcement | LLM Stats |
| Nova 2 Omni | 0.614 | Not recorded | Not recorded | 2025-12-02 · model_announcement | LLM Stats |
| Qwen3 VL 30B A3B Instruct | 0.604 | Not recorded | Not recorded | 2025-09-22 · model_announcement | LLM Stats |
| Qwen3 VL 8B Thinking | 0.604 | Not recorded | Not recorded | 2025-09-22 · model_announcement | LLM Stats |
| Mistral Small 4 | 0.6 | Not recorded | Not recorded | 2026-03-16 · model_announcement | LLM Stats |
| GPT-4o | 0.599 | Not recorded | Not recorded | 2024-08-06 · model_announcement | LLM Stats |
| Llama 4 Maverick | 0.596 | Not recorded | Not recorded | 2025-04-05 · model_announcement | LLM Stats |
| Qwen3 VL 4B Thinking | 0.57 | Not recorded | Not recorded | 2025-09-22 · model_announcement | LLM Stats |
| Qwen3 VL 8B Instruct | 0.559 | Not recorded | Not recorded | 2025-09-22 · model_announcement | LLM Stats |
| DiffusionGemma 26B-A4B | 0.543 | Not recorded | Not recorded | 2026-06-10 · model_announcement | LLM Stats |
| Qwen3 VL 4B Instruct | 0.532 | Not recorded | Not recorded | 2025-09-22 · model_announcement | LLM Stats |
| Gemma 4 E4B | 0.526 | Not recorded | Not recorded | 2026-04-02 · model_announcement | LLM Stats |
| Qwen2.5 VL 72B Instruct | 0.511 | Not recorded | Not recorded | 2025-01-26 · model_announcement | LLM Stats |
| Qwen2.5 VL 32B Instruct | 0.495 | Not recorded | Not recorded | 2025-02-28 · model_announcement | LLM Stats |
| Qwen2-VL-72B-Instruct | 0.462 | Not recorded | Not recorded | 2024-08-29 · model_announcement | LLM Stats |
| Llama 3.2 90B Instruct | 0.452 | Not recorded | Not recorded | 2024-09-25 · model_announcement | LLM Stats |
| Gemma 4 E2B | 0.442 | Not recorded | Not recorded | 2026-04-02 · model_announcement | LLM Stats |
| Phi-4-multimodal-instruct | 0.385 | Not recorded | Not recorded | 2025-02-01 · model_announcement | LLM Stats |
| Qwen2.5 VL 7B Instruct | 0.383 | Not recorded | Not recorded | 2025-01-26 · model_announcement | LLM Stats |
| Qwen2.5-Omni-7B | 0.366 | Not recorded | Not recorded | 2025-03-27 · model_announcement | LLM Stats |
| Llama 3.2 11B Instruct | 0.33 | Not recorded | Not recorded | 2024-09-25 · model_announcement | LLM Stats |
| LFM2.5-VL-3B | 0.305 | Not recorded | Not recorded | 2026-08-12 · model_announcement | LLM Stats |