AI EVALUATION, IN CONTEXT.BY EQUALITY MACHINE
Benchmark Radar.Benchmarks. Model results. Original sources.
/

Artificial Analysis

MMMU-Pro

Visual reasoning

Evaluation context and evidence

Source
Artificial Analysis
Publisher
Not recorded
Benchmark release
2024-09-05
Collected
2026-08-25
Modality
Not recorded

These are source-reported observations, not independent reruns by this site. Check the task version, evaluation protocol, unit and reporting date before comparing models. Results from different setups are not a universal ranking.

Original sources

Reported results 245

Shown in source observation order, not ranked. Dates retain the precision provided by the source.

ModelValueUnitProtocolReported date and basisSource
Gemini 3.7 Flash (high)0.854913294797688Not recordedNot recorded2026-08-13 · model_announcementArtificial Analysis
Gemini 3.7 Flash (low)0.848554913294798Not recordedNot recorded2026-08-13 · model_announcementArtificial Analysis
Claude Opus 5 (Adaptive Reasoning, Max Effort)0.847398843930636Not recordedNot recorded2026-07-24 · model_announcementArtificial Analysis
Gemini 3.7 Flash (medium)0.847398843930636Not recordedNot recorded2026-08-13 · model_announcementArtificial Analysis
Gemini 3.5 Flash (high)0.842774566473988Not recordedNot recorded2026-05-19 · model_announcementArtificial Analysis
Claude Opus 5 (Adaptive Reasoning, Xhigh Effort)0.839884393063584Not recordedNot recorded2026-07-24 · model_announcementArtificial Analysis
Gemini 3.5 Flash (medium)0.838728323699422Not recordedNot recorded2026-05-19 · model_announcementArtificial Analysis
GPT-5.6 Sol (max)0.834104046242775Not recordedNot recorded2026-07-09 · model_announcementArtificial Analysis
Gemini 3.6 Flash (high)0.832369942196532Not recordedNot recorded2026-07-21 · model_announcementArtificial Analysis
GPT-5.6 Sol (xhigh)0.826589595375723Not recordedNot recorded2026-07-09 · model_announcementArtificial Analysis
Claude Opus 5 (Adaptive Reasoning, High Effort)0.824277456647399Not recordedNot recorded2026-07-24 · model_announcementArtificial Analysis
Gemini 3.1 Pro Preview0.824277456647399Not recordedNot recorded2026-02-19 · model_announcementArtificial Analysis
Qwen3.8 Max0.823121387283237Not recordedNot recorded2026-08-03 · model_announcementArtificial Analysis
GPT-5.6 Sol (high)0.81849710982659Not recordedNot recorded2026-07-09 · model_announcementArtificial Analysis
Claude Opus 5 (Adaptive Reasoning, Medium Effort)0.816184971098266Not recordedNot recorded2026-07-24 · model_announcementArtificial Analysis
GPT-5.6 Sol (medium)0.813872832369942Not recordedNot recorded2026-07-09 · model_announcementArtificial Analysis
GPT-5.5 (medium)0.811560693641619Not recordedNot recorded2026-04-23 · model_announcementArtificial Analysis
GPT-5.5 (high)0.810982658959538Not recordedNot recorded2026-04-23 · model_announcementArtificial Analysis
GPT-5.6 Sol (low)0.809826589595376Not recordedNot recorded2026-07-09 · model_announcementArtificial Analysis
GPT-5.6 Terra (max)0.806936416184971Not recordedNot recorded2026-07-09 · model_announcementArtificial Analysis
Muse Spark0.805202312138728Not recordedNot recorded2026-04-08 · model_announcementArtificial Analysis
Kimi K3 (max)0.805202312138728Not recordedNot recorded2026-07-16 · model_announcementArtificial Analysis
Qwen3.7 Plus0.804624277456647Not recordedNot recorded2026-06-01 · model_announcementArtificial Analysis
Grok 4.5 (high)0.804046242774566Not recordedNot recorded2026-07-08 · model_announcementArtificial Analysis
Gemini 3 Pro Preview (high)0.801734104046243Not recordedNot recorded2025-11-18 · model_announcementArtificial Analysis
Gemini 3.5 Flash (minimal)0.801156069364162Not recordedNot recorded2026-05-19 · model_announcementArtificial Analysis
Gemini 3 Flash Preview (Reasoning)0.799421965317919Not recordedNot recorded2025-12-17 · model_announcementArtificial Analysis
GPT-5.5 (xhigh)0.798843930635838Not recordedNot recorded2026-04-23 · model_announcementArtificial Analysis
Claude Opus 5 (Adaptive Reasoning, Low Effort)0.798265895953757Not recordedNot recorded2026-07-24 · model_announcementArtificial Analysis
GPT-5.6 Terra (xhigh)0.794797687861272Not recordedNot recorded2026-07-09 · model_announcementArtificial Analysis
Kimi K2.60.79364161849711Not recordedNot recorded2026-04-20 · model_announcementArtificial Analysis
GPT-5.6 Terra (high)0.790751445086705Not recordedNot recorded2026-07-09 · model_announcementArtificial Analysis
Gemini 3.5 Flash-Lite0.790173410404624Not recordedNot recorded2026-07-21 · model_announcementArtificial Analysis
GPT-5.5 (low)0.790173410404624Not recordedNot recorded2026-04-23 · model_announcementArtificial Analysis
Claude Opus 4.7 (Adaptive Reasoning, Max Effort)0.788439306358382Not recordedNot recorded2026-04-16 · model_announcementArtificial Analysis
MiniMax-M30.785549132947977Not recordedNot recorded2026-06-01 · model_announcementArtificial Analysis
GPT-5.6 Luna (max)0.785549132947977Not recordedNot recorded2026-07-09 · model_announcementArtificial Analysis
Gemini 3 Flash Preview (Non-reasoning)0.785549132947977Not recordedNot recorded2025-12-17 · model_announcementArtificial Analysis
GPT-5.6 Luna (xhigh)0.785549132947977Not recordedNot recorded2026-07-09 · model_announcementArtificial Analysis
Kimi K3 (low)0.784971098265896Not recordedNot recorded2026-07-16 · model_announcementArtificial Analysis
GPT-5.3 Codex (xhigh)0.784971098265896Not recordedNot recorded2026-02-05 · model_announcementArtificial Analysis
GPT-5.4 (xhigh)0.784393063583815Not recordedNot recorded2026-03-05 · model_announcementArtificial Analysis
Grok 4.3 (high)0.78092485549133Not recordedNot recorded2026-04-30 · model_announcementArtificial Analysis
GPT-5.4 (low)0.779768786127168Not recordedNot recorded2026-03-05 · model_announcementArtificial Analysis
Qwen3.6 Plus0.779768786127168Not recordedNot recorded2026-04-02 · model_announcementArtificial Analysis
GPT-5.6 Luna (high)0.775722543352601Not recordedNot recorded2026-07-09 · model_announcementArtificial Analysis
Qwen3.5 397B A17B (Reasoning)0.772832369942197Not recordedNot recorded2026-02-16 · model_announcementArtificial Analysis
Claude Sonnet 5 (Adaptive Reasoning, Max Effort)0.772832369942197Not recordedNot recorded2026-06-30 · model_announcementArtificial Analysis
GPT-5.6 Terra (medium)0.767630057803468Not recordedNot recorded2026-07-09 · model_announcementArtificial Analysis
Grok Build 0.1 06160.765317919075144Not recordedNot recorded2026-06-16 · model_announcementArtificial Analysis
Claude Opus 4.7 (Non-reasoning, High Effort)0.763583815028902Not recordedNot recorded2026-04-16 · model_announcementArtificial Analysis
GPT-5.2 Codex (xhigh)0.763005780346821Not recordedNot recorded2025-12-11 · model_announcementArtificial Analysis
Qwen3.8 27B (xhigh)0.763005780346821Not recordedNot recorded2026-08-14 · model_announcementArtificial Analysis
GPT-5.6 Terra (low)0.761271676300578Not recordedNot recorded2026-07-09 · model_announcementArtificial Analysis
GPT-5.6 Luna (medium)0.758381502890173Not recordedNot recorded2026-07-09 · model_announcementArtificial Analysis
Grok 4.3 (medium)0.758381502890173Not recordedNot recorded2026-04-30 · model_announcementArtificial Analysis
Gemini 3.1 Flash-Lite0.755491329479769Not recordedNot recorded2026-03-03 · model_announcementArtificial Analysis
GPT-5.1 (high)0.754913294797688Not recordedNot recorded2025-11-13 · model_announcementArtificial Analysis
MiMo-V2.50.754335260115607Not recordedNot recorded2026-04-22 · model_announcementArtificial Analysis
Claude Opus 4.6 (Adaptive Reasoning, Max Effort)0.754335260115607Not recordedNot recorded2026-02-05 · model_announcementArtificial Analysis
Kimi K2.5 (Reasoning)0.753757225433526Not recordedNot recorded2026-01-27 · model_announcementArtificial Analysis
Step 3.7 Flash0.753179190751445Not recordedNot recorded2026-05-29 · model_announcementArtificial Analysis
Qwen3.5 27B (Reasoning)0.75028901734104Not recordedNot recorded2026-02-24 · model_announcementArtificial Analysis
Qwen3.6 35B A3B (Reasoning)0.75028901734104Not recordedNot recorded2026-04-16 · model_announcementArtificial Analysis
Qwen3.5 122B A10B (Reasoning)0.74971098265896Not recordedNot recorded2026-02-24 · model_announcementArtificial Analysis
Gemini 2.5 Pro0.749132947976879Not recordedNot recorded2025-06-05 · model_announcementArtificial Analysis
DeepSeek V4 Flash Vision (Reasoning, Max Effort)0.747976878612717Not recordedNot recorded2026-08-21 · model_announcementArtificial Analysis
Qwen3.6 27B (Reasoning)0.746242774566474Not recordedNot recorded2026-04-22 · model_announcementArtificial Analysis
GPT-5.2 (medium)0.745664739884393Not recordedNot recorded2025-12-11 · model_announcementArtificial Analysis
Grok 4.20 0309 v2 (Reasoning)0.745664739884393Not recordedNot recorded2026-04-07 · model_announcementArtificial Analysis
Muse Glimmer (high)0.743352601156069Not recordedNot recorded2026-08-10 · model_announcementArtificial Analysis
GPT-5 (medium)0.743352601156069Not recordedNot recorded2025-08-07 · model_announcementArtificial Analysis
GPT-5 (high)0.742196531791907Not recordedNot recorded2025-08-07 · model_announcementArtificial Analysis
Qwen3.8 27B (medium)0.741618497109827Not recordedNot recorded2026-08-14 · model_announcementArtificial Analysis
Claude Opus 4.5 (Reasoning)0.740462427745665Not recordedNot recorded2025-11-24 · model_announcementArtificial Analysis
Inkling Small0.740462427745665Not recordedNot recorded2026-07-30 · model_announcementArtificial Analysis
GPT-5.6 Luna (low)0.739884393063584Not recordedNot recorded2026-07-09 · model_announcementArtificial Analysis
MiMo-V2-Omni-03270.739306358381503Not recordedNot recorded2026-03-27 · model_announcementArtificial Analysis
GPT-5 Codex (high)0.738150289017341Not recordedNot recorded2025-09-23 · model_announcementArtificial Analysis
Qwen3.8 27B (low)0.73757225433526Not recordedNot recorded2026-08-14 · model_announcementArtificial Analysis
GPT-5 (low)0.73757225433526Not recordedNot recorded2025-08-07 · model_announcementArtificial Analysis
Inkling (xhigh)0.734682080924856Not recordedNot recorded2026-07-15 · model_announcementArtificial Analysis
Gemma 4 31B (Reasoning)0.734104046242775Not recordedNot recorded2026-04-02 · model_announcementArtificial Analysis
GPT-5.4 mini (xhigh)0.732947976878613Not recordedNot recorded2026-03-17 · model_announcementArtificial Analysis
Claude Sonnet 4.6 (Adaptive Reasoning, Max Effort)0.732947976878613Not recordedNot recorded2026-02-17 · model_announcementArtificial Analysis
Grok 4.20 0309 (Reasoning)0.731791907514451Not recordedNot recorded2026-03-10 · model_announcementArtificial Analysis
Gemini 2.5 Flash Preview (Sep '25) (Reasoning)0.73121387283237Not recordedNot recorded2025-09-25 · model_announcementArtificial Analysis
Kimi K2.5 (Non-reasoning)0.730635838150289Not recordedNot recorded2026-01-27 · model_announcementArtificial Analysis
GLM 5V Turbo (Reasoning)0.727745664739884Not recordedNot recorded2026-04-01 · model_announcementArtificial Analysis
Grok 4.3 (low)0.727745664739884Not recordedNot recorded2026-04-30 · model_announcementArtificial Analysis
Qwen3.5 35B A3B (Reasoning)0.726589595375723Not recordedNot recorded2026-02-24 · model_announcementArtificial Analysis
Claude Opus 4.6 (Non-reasoning, High Effort)0.725433526011561Not recordedNot recorded2026-02-05 · model_announcementArtificial Analysis
GPT-5.1 Codex (high)0.72485549132948Not recordedNot recorded2025-11-13 · model_announcementArtificial Analysis
GPT-5.6 Sol (Non-reasoning)0.71907514450867Not recordedNot recorded2026-07-09 · model_announcementArtificial Analysis
Claude Sonnet 5 (Non-reasoning, High Effort)0.71907514450867Not recordedNot recorded2026-06-30 · model_announcementArtificial Analysis
Qwen3.6 27B (Non-reasoning)0.717341040462428Not recordedNot recorded2026-04-22 · model_announcementArtificial Analysis
GPT-5.5 (Non-reasoning)0.713872832369942Not recordedNot recorded2026-04-23 · model_announcementArtificial Analysis
Claude Opus 4.5 (Non-reasoning)0.712138728323699Not recordedNot recorded2025-11-24 · model_announcementArtificial Analysis
GPT-5.4 mini (medium)0.711560693641618Not recordedNot recorded2026-03-17 · model_announcementArtificial Analysis
Qwen3.6 35B A3B (Non-reasoning)0.710404624277457Not recordedNot recorded2026-04-16 · model_announcementArtificial Analysis
Claude Sonnet 4.6 (Non-reasoning, High Effort)0.705780346820809Not recordedNot recorded2026-02-17 · model_announcementArtificial Analysis
GPT-5.4 (Non-reasoning)0.705780346820809Not recordedNot recorded2026-03-05 · model_announcementArtificial Analysis
Qwen3.5 Omni Plus0.705202312138728Not recordedNot recorded2026-03-30 · model_announcementArtificial Analysis
Gemma 4 31B (Non-reasoning)0.703468208092486Not recordedNot recorded2026-04-02 · model_announcementArtificial Analysis
Qwen3.5 122B A10B (Non-reasoning)0.702890173410405Not recordedNot recorded2026-02-24 · model_announcementArtificial Analysis
Gemini 2.5 Flash Preview (Sep '25) (Non-reasoning)0.702312138728324Not recordedNot recorded2025-09-25 · model_announcementArtificial Analysis
o30.700578034682081Not recordedNot recorded2025-04-16 · model_announcementArtificial Analysis
GPT-5 mini (high)0.700578034682081Not recordedNot recorded2025-08-07 · model_announcementArtificial Analysis
Qwen3.5 27B (Non-reasoning)0.7Not recordedNot recorded2026-02-24 · model_announcementArtificial Analysis
Qwen3.8 27B (Non-reasoning)0.699421965317919Not recordedNot recorded2026-08-14 · model_announcementArtificial Analysis
MiMo-V2-Omni0.698843930635838Not recordedNot recorded2026-03-19 · model_announcementArtificial Analysis
Gemma 4 12B (Reasoning)0.696531791907514Not recordedNot recorded2026-06-03 · model_announcementArtificial Analysis
Qwen3.5 9B (Reasoning)0.692485549132948Not recordedNot recorded2026-03-02 · model_announcementArtificial Analysis
Gemma 4 26B A4B (Reasoning)0.692485549132948Not recordedNot recorded2026-04-02 · model_announcementArtificial Analysis
o4-mini (high)0.692485549132948Not recordedNot recorded2025-04-16 · model_announcementArtificial Analysis
Qwen3.5 35B A3B (Non-reasoning)0.691907514450867Not recordedNot recorded2026-02-24 · model_announcementArtificial Analysis
Claude Sonnet 4.6 (Non-reasoning, Low Effort)0.691907514450867Not recordedNot recorded2026-02-17 · model_announcementArtificial Analysis
Gemini 2.5 Flash (Reasoning)0.690751445086705Not recordedNot recorded2025-05-20 · model_announcementArtificial Analysis
GPT-5.1 Codex mini (high)0.690173410404624Not recordedNot recorded2025-11-13 · model_announcementArtificial Analysis
Grok 40.688439306358381Not recordedNot recorded2025-07-10 · model_announcementArtificial Analysis
GPT-5 mini (medium)0.688439306358381Not recordedNot recorded2025-08-07 · model_announcementArtificial Analysis
Claude 4.5 Sonnet (Reasoning)0.68728323699422Not recordedNot recorded2025-09-29 · model_announcementArtificial Analysis
Qwen3 VL 235B A22B (Reasoning)0.68728323699422Not recordedNot recorded2025-09-23 · model_announcementArtificial Analysis
Doubao Seed Code0.68092485549133Not recordedNot recorded2025-11-11 · model_announcementArtificial Analysis
Claude 4.1 Opus (Reasoning)0.679190751445087Not recordedNot recorded2025-08-05 · model_announcementArtificial Analysis
Qwen3 VL 235B A22B Instruct0.675722543352601Not recordedNot recorded2025-09-23 · model_announcementArtificial Analysis
EXAONE 4.5 33B0.673410404624277Not recordedNot recorded2026-04-09 · model_announcementArtificial Analysis
Qwen3.5 9B (Non-reasoning)0.667630057803468Not recordedNot recorded2026-03-02 · model_announcementArtificial Analysis
Gemma 4 26B A4B (Non-reasoning)0.667052023121387Not recordedNot recorded2026-04-02 · model_announcementArtificial Analysis
GPT-5.6 Terra (Non-reasoning)0.667052023121387Not recordedNot recorded2026-07-09 · model_announcementArtificial Analysis
DiffusionGemma 26B A4B0.665317919075144Not recordedNot recorded2026-06-10 · model_announcementArtificial Analysis
GPT-5.2 (Non-reasoning)0.658381502890173Not recordedNot recorded2025-12-11 · model_announcementArtificial Analysis
Gemini 2.5 Flash (Non-reasoning)0.654913294797688Not recordedNot recorded2025-05-20 · model_announcementArtificial Analysis
Qwen3.5 4B (Reasoning)0.653757225433526Not recordedNot recorded2026-03-02 · model_announcementArtificial Analysis
GPT-5.4 nano (xhigh)0.653757225433526Not recordedNot recorded2026-03-17 · model_announcementArtificial Analysis
Claude 4.5 Sonnet (Non-reasoning)0.652023121387283Not recordedNot recorded2025-09-29 · model_announcementArtificial Analysis
Gemini 2.5 Flash-Lite Preview (Sep '25) (Reasoning)0.64971098265896Not recordedNot recorded2025-09-08 · model_announcementArtificial Analysis
Grok 4.20 0309 v2 (Non-reasoning)0.649132947976879Not recordedNot recorded2026-04-07 · model_announcementArtificial Analysis
Mistral Medium 3.50.648554913294798Not recordedNot recorded2026-04-29 · model_announcementArtificial Analysis
Grok 4.3 (Non-reasoning)0.647976878612717Not recordedNot recorded2026-04-30 · model_announcementArtificial Analysis
Qwen3.5 Omni Flash0.647398843930636Not recordedNot recorded2026-03-30 · model_announcementArtificial Analysis
ERNIE 5.0 Thinking Preview0.645664739884393Not recordedNot recorded2025-11-13 · model_announcementArtificial Analysis
Nova 2.0 Pro Preview (medium)0.645086705202312Not recordedNot recorded2025-11-27 · model_announcementArtificial Analysis
Qwen3 VL 32B Instruct0.642774566473988Not recordedNot recorded2025-10-21 · model_announcementArtificial Analysis
JT-4.1 Flash 236B A21B0.641040462427746Not recordedNot recorded2026-07-09 · model_announcementArtificial Analysis
Grok 4.20 0309 (Non-reasoning)0.640462427745665Not recordedNot recorded2026-03-10 · model_announcementArtificial Analysis
Step3 VL 10B0.639884393063584Not recordedNot recorded2026-01-20 · model_announcementArtificial Analysis
Nova 2.0 Lite (high)0.63757225433526Not recordedNot recorded2025-10-29 · model_announcementArtificial Analysis
Gemini 2.5 Flash-Lite Preview (Sep '25) (Non-reasoning)0.634104046242775Not recordedNot recorded2025-09-25 · model_announcementArtificial Analysis
Qwen3 VL 32B (Reasoning)0.634104046242775Not recordedNot recorded2025-10-21 · model_announcementArtificial Analysis
Grok 4.1 Fast (Reasoning)0.632947976878613Not recordedNot recorded2025-11-19 · model_announcementArtificial Analysis
Command A+0.632369942196532Not recordedNot recorded2026-05-20 · model_announcementArtificial Analysis
Nova 2.0 Pro Preview (low)0.627167630057804Not recordedNot recorded2025-11-27 · model_announcementArtificial Analysis
Nova 2.0 Lite (medium)0.625433526011561Not recordedNot recorded2025-10-29 · model_announcementArtificial Analysis
Claude 4 Sonnet (Non-reasoning)0.623699421965318Not recordedNot recorded2025-05-22 · model_announcementArtificial Analysis
GPT-5.1 (Non-reasoning)0.623699421965318Not recordedNot recorded2025-11-13 · model_announcementArtificial Analysis
Qwen3 VL 30B A3B Instruct0.621387283236994Not recordedNot recorded2025-10-03 · model_announcementArtificial Analysis
Llama 4 Maverick0.621387283236994Not recordedNot recorded2025-04-05 · model_announcementArtificial Analysis
Qwen3.5 4B (Non-reasoning)0.620809248554913Not recordedNot recorded2026-03-02 · model_announcementArtificial Analysis
GPT-5 (minimal)0.620809248554913Not recordedNot recorded2025-08-07 · model_announcementArtificial Analysis
Gemma 4 12B (Non-reasoning)0.619653179190751Not recordedNot recorded2026-06-03 · model_announcementArtificial Analysis
Gemini 2.5 Flash Preview (Non-reasoning)0.619653179190751Not recordedNot recorded2025-04-17 · model_announcementArtificial Analysis
Nova 2.0 Omni (medium)0.619075144508671Not recordedNot recorded2025-11-26 · model_announcementArtificial Analysis
Qwen3 VL 30B A3B (Reasoning)0.61849710982659Not recordedNot recorded2025-10-03 · model_announcementArtificial Analysis
Grok 4 Fast (Reasoning)0.617919075144509Not recordedNot recorded2025-09-19 · model_announcementArtificial Analysis
Claude 4 Sonnet (Reasoning)0.617919075144509Not recordedNot recorded2025-05-22 · model_announcementArtificial Analysis
GPT-4.10.612138728323699Not recordedNot recorded2025-04-14 · model_announcementArtificial Analysis
GPT-5 nano (high)0.609826589595376Not recordedNot recorded2025-08-07 · model_announcementArtificial Analysis
GPT-5.4 mini (Non-Reasoning)0.604624277456647Not recordedNot recorded2026-03-17 · model_announcementArtificial Analysis
GPT-5.6 Luna (Non-reasoning)0.603468208092486Not recordedNot recorded2026-07-09 · model_announcementArtificial Analysis
Qwen3 Omni 30B A3B (Reasoning)0.602312138728324Not recordedNot recorded2025-09-22 · model_announcementArtificial Analysis
Claude 3.7 Sonnet (Non-reasoning)0.600578034682081Not recordedNot recorded2025-02-24 · model_announcementArtificial Analysis
Nova 2.0 Omni (low)0.598265895953757Not recordedNot recorded2025-11-26 · model_announcementArtificial Analysis
Magistral Medium 1.20.596531791907514Not recordedNot recorded2025-09-18 · model_announcementArtificial Analysis
GPT-5.4 nano (medium)0.595375722543353Not recordedNot recorded2026-03-17 · model_announcementArtificial Analysis
GPT-4.1 mini0.58728323699422Not recordedNot recorded2025-04-14 · model_announcementArtificial Analysis
Claude 4.5 Haiku (Reasoning)0.585549132947977Not recordedNot recorded2025-10-15 · model_announcementArtificial Analysis
GPT-5 mini (minimal)0.583815028901734Not recordedNot recorded2025-08-07 · model_announcementArtificial Analysis
GPT-5 nano (medium)0.582080924855491Not recordedNot recorded2025-08-07 · model_announcementArtificial Analysis
Gemini 2.5 Flash-Lite (Reasoning)0.582080924855491Not recordedNot recorded2025-06-17 · model_announcementArtificial Analysis
Nova 2.0 Lite (low)0.579768786127168Not recordedNot recorded2025-10-29 · model_announcementArtificial Analysis
Apriel-v1.5-15B-Thinker0.570520231213873Not recordedNot recorded2025-09-30 · model_announcementArtificial Analysis
Mistral Small 4 (Reasoning)0.568208092485549Not recordedNot recorded2026-03-16 · model_announcementArtificial Analysis
Qwen3 VL 8B (Reasoning)0.566473988439306Not recordedNot recorded2025-10-14 · model_announcementArtificial Analysis
GPT-4o (Aug '24)0.563005780346821Not recordedNot recorded2024-08-06 · model_announcementArtificial Analysis
Mistral Large 30.556647398843931Not recordedNot recorded2025-12-02 · model_announcementArtificial Analysis
Qwen3 Omni 30B A3B Instruct0.554913294797688Not recordedNot recorded2025-09-22 · model_announcementArtificial Analysis
Magistral Small 1.20.554913294797688Not recordedNot recorded2025-09-17 · model_announcementArtificial Analysis
Claude 4.5 Haiku (Non-reasoning)0.551445086705202Not recordedNot recorded2025-10-15 · model_announcementArtificial Analysis
Gemini 1.5 Pro (Sep '24)0.55028901734104Not recordedNot recorded2024-09-24 · model_announcementArtificial Analysis
Mistral Medium 3.10.541618497109827Not recordedNot recorded2025-08-12 · model_announcementArtificial Analysis
Gemini 2.5 Flash-Lite (Non-reasoning)0.539884393063584Not recordedNot recorded2025-06-17 · model_announcementArtificial Analysis
Nemotron 3 Nano Omni 30B A3B Reasoning0.531791907514451Not recordedNot recorded2026-04-29 · model_announcementArtificial Analysis
Mistral Medium 30.530057803468208Not recordedNot recorded2025-05-07 · model_announcementArtificial Analysis
Llama 4 Scout0.529479768786127Not recordedNot recorded2025-04-05 · model_announcementArtificial Analysis
NVIDIA Nemotron Nano 12B v2 VL (Reasoning)0.528901734104046Not recordedNot recorded2025-10-28 · model_announcementArtificial Analysis
Qwen3.5 397B A17B (Non-reasoning)0.526589595375723Not recordedNot recorded2026-02-16 · model_announcementArtificial Analysis
Qwen3 VL 4B (Reasoning)0.520231213872832Not recordedNot recorded2025-10-14 · model_announcementArtificial Analysis
Gemma 4 E4B (Reasoning)0.513872832369942Not recordedNot recorded2026-04-03 · model_announcementArtificial Analysis
Gemma 4 E4B (Non-reasoning)0.512138728323699Not recordedNot recorded2026-04-03 · model_announcementArtificial Analysis
Pixtral Large0.505780346820809Not recordedNot recorded2024-11-18 · model_announcementArtificial Analysis
GLM-4.5V (Reasoning)0.504624277456647Not recordedNot recorded2025-08-11 · model_announcementArtificial Analysis
Nova 2.0 Omni (Non-reasoning)0.498843930635838Not recordedNot recorded2025-11-26 · model_announcementArtificial Analysis
Ministral 3 14B0.498265895953757Not recordedNot recorded2025-12-02 · model_announcementArtificial Analysis
Nova 2.0 Lite (Non-reasoning)0.490173410404624Not recordedNot recorded2025-10-29 · model_announcementArtificial Analysis
GLM-4.6V (Reasoning)0.485549132947977Not recordedNot recorded2025-12-08 · model_announcementArtificial Analysis
Grok 4.1 Fast (Non-reasoning)0.484393063583815Not recordedNot recorded2025-11-19 · model_announcementArtificial Analysis
Gemini 1.5 Flash (Sep '24)0.483815028901734Not recordedNot recorded2024-09-24 · model_announcementArtificial Analysis
Grok 4 Fast (Non-reasoning)0.480924855491329Not recordedNot recorded2025-09-19 · model_announcementArtificial Analysis
Gemma 3 27B Instruct0.480346820809249Not recordedNot recorded2025-03-12 · model_announcementArtificial Analysis
Mistral Small 3.20.479768786127168Not recordedNot recorded2025-06-20 · model_announcementArtificial Analysis
Qwen3 VL 8B Instruct0.473410404624277Not recordedNot recorded2025-10-14 · model_announcementArtificial Analysis
Mistral Small 4 (Non-reasoning)0.464739884393064Not recordedNot recorded2026-03-16 · model_announcementArtificial Analysis
Ministral 3 8B0.459537572254335Not recordedNot recorded2025-12-02 · model_announcementArtificial Analysis
Claude 3.5 Haiku0.45606936416185Not recordedNot recorded2024-10-22 · model_announcementArtificial Analysis
Devstral Small 20.446242774566474Not recordedNot recorded2025-12-09 · model_announcementArtificial Analysis
Gemma 4 E2B (Reasoning)0.445664739884393Not recordedNot recorded2026-04-02 · model_announcementArtificial Analysis
NVIDIA Nemotron Nano 12B v2 VL (Non-reasoning)0.445086705202312Not recordedNot recorded2025-10-28 · model_announcementArtificial Analysis
Nova Pro0.443352601156069Not recordedNot recorded2024-12-03 · model_announcementArtificial Analysis
Qwen3 VL 4B Instruct0.438728323699422Not recordedNot recorded2025-10-14 · model_announcementArtificial Analysis
GPT-5.4 nano (Non-Reasoning)0.43757225433526Not recordedNot recorded2026-03-17 · model_announcementArtificial Analysis
Qwen3.5 2B (Reasoning)0.430057803468208Not recordedNot recorded2026-03-02 · model_announcementArtificial Analysis
GLM-4.5V (Non-reasoning)0.427745664739884Not recordedNot recorded2025-08-11 · model_announcementArtificial Analysis
Qwen3.5 2B (Non-reasoning)0.426589595375723Not recordedNot recorded2026-03-02 · model_announcementArtificial Analysis
GLM-4.6V (Non-reasoning)0.421965317919075Not recordedNot recorded2025-12-08 · model_announcementArtificial Analysis
Gemma 4 E2B (Non-reasoning)0.417919075144509Not recordedNot recorded2026-04-02 · model_announcementArtificial Analysis
GPT-4o mini0.415028901734104Not recordedNot recorded2024-07-18 · model_announcementArtificial Analysis
GPT-4.1 nano0.401156069364162Not recordedNot recorded2025-04-14 · model_announcementArtificial Analysis
Llama 3.2 Instruct 90B (Vision)0.394797687861272Not recordedNot recorded2024-09-25 · model_announcementArtificial Analysis
Ministral 3 3B0.380924855491329Not recordedNot recorded2025-12-02 · model_announcementArtificial Analysis
MiniCPM-V 4.6 1.3B0.379190751445087Not recordedNot recorded2026-05-11 · model_announcementArtificial Analysis
Nova Lite0.378034682080925Not recordedNot recorded2024-12-03 · model_announcementArtificial Analysis
Gemma 3 12B Instruct0.37514450867052Not recordedNot recorded2025-03-12 · model_announcementArtificial Analysis
Molmo2-8B0.374566473988439Not recordedNot recorded2025-12-11 · model_announcementArtificial Analysis
Gemini 1.5 Flash-8B0.365317919075144Not recordedNot recorded2024-10-03 · model_announcementArtificial Analysis
GPT-5 nano (minimal)0.317919075144509Not recordedNot recorded2025-08-07 · model_announcementArtificial Analysis
Claude 3 Haiku0.308092485549133Not recordedNot recorded2024-03-04 · model_announcementArtificial Analysis
Gemma 3 4B Instruct0.299421965317919Not recordedNot recorded2025-03-12 · model_announcementArtificial Analysis
Llama 3.2 Instruct 11B (Vision)0.293063583815029Not recordedNot recorded2024-09-25 · model_announcementArtificial Analysis
LFM2.5-VL-1.6B0.265317919075145Not recordedNot recorded2026-01-05 · model_announcementArtificial Analysis
Gemma 3n E4B Instruct0.26242774566474Not recordedNot recorded2025-06-26 · model_announcementArtificial Analysis
Qwen3.5 0.8B (Reasoning)0.258381502890173Not recordedNot recorded2026-03-02 · model_announcementArtificial Analysis
Qwen3.5 0.8B (Non-reasoning)0.256647398843931Not recordedNot recorded2026-03-02 · model_announcementArtificial Analysis
Molmo 7B-D0.245086705202312Not recordedNot recorded2024-09-25 · model_announcementArtificial Analysis
Phi-4 Multimodal Instruct0.145086705202312Not recordedNot recorded2025-02-26 · model_announcementArtificial Analysis

Put this benchmark in context

Multimodal benchmarksReasoning and math benchmarks