AI benchmarks and leaderboards are different things
Learn the difference between a benchmark, an evaluation run and a leaderboard, and use each to answer the right question.
LLM Stats
Video-MME
Video-MME is the first-ever comprehensive evaluation benchmark of Multi-modal Large Language Models (MLLMs) in video analysis. It features 900 videos totaling 254 hours with 2,700 human-annotated question-answer pairs across 6 primary visual domains (Knowledge, Film & Television, Sports Competition, Life Record, Multilingual, and others) and 30 subfields. The benchmark evaluates models across diverse temporal dimensions (11 seconds to 1 hour), integrates multi-modal inputs including video frames, subtitles, and audio, and uses rigorous manual labeling by expert annotators for precise assessment.
Evaluation context and evidence
- Source
- LLM Stats
- Publisher
- Not recorded
- Benchmark release
- Not recorded
- Collected
- 2026-08-17
- Modality
- multimodal
These are source-reported observations, not independent reruns by this site. Check the task version, evaluation protocol, unit and reporting date before comparing models. Results from different setups are not a universal ranking.
Original sources
Reported results 17
Shown in source observation order, not ranked. Dates retain the precision provided by the source.
| Model | Value | Unit | Protocol | Reported date and basis | Source |
|---|---|---|---|---|---|
| Seed 2.1 Pro | 0.892 | Not recorded | Not recorded | 2026-06-24 · model_announcement | LLM Stats |
| Seed 2.1 Turbo | 0.89 | Not recorded | Not recorded | 2026-06-24 · model_announcement | LLM Stats |
| Qwen3.7-Plus | 0.88 | Not recorded | Not recorded | 2026-05-31 · model_announcement | LLM Stats |
| MiMo-V2.5 | 0.877 | Not recorded | Not recorded | 2026-04-22 · model_announcement | LLM Stats |
| Kimi K2.5 | 0.874 | Not recorded | Not recorded | 2026-01-27 · model_announcement | LLM Stats |
| MiniMax M3 | 0.854 | Not recorded | Not recorded | 2026-06-01 · model_announcement | LLM Stats |
| Gemini 2.5 Pro | 0.848 | Not recorded | Not recorded | 2025-05-20 · model_announcement | LLM Stats |
| Qwen3.6 Plus | 0.842 | Not recorded | Not recorded | 2026-04-02 · model_announcement | LLM Stats |
| Gemini 1.5 Pro | 0.786 | Not recorded | Not recorded | 2024-05-01 · model_announcement | LLM Stats |
| Nova 2 Omni | 0.779 | Not recorded | Not recorded | 2025-12-02 · model_announcement | LLM Stats |
| Gemini 1.5 Flash | 0.761 | Not recorded | Not recorded | 2024-05-01 · model_announcement | LLM Stats |
| Qwen3 VL 30B A3B Instruct | 0.745 | Not recorded | Not recorded | 2025-09-22 · model_announcement | LLM Stats |
| Qwen3 VL 30B A3B Thinking | 0.733 | Not recorded | Not recorded | 2025-09-22 · model_announcement | LLM Stats |
| Qwen3 VL 8B Thinking | 0.718 | Not recorded | Not recorded | 2025-09-22 · model_announcement | LLM Stats |
| Qwen3 VL 8B Instruct | 0.714 | Not recorded | Not recorded | 2025-09-22 · model_announcement | LLM Stats |
| Gemini 1.5 Flash 8B | 0.662 | Not recorded | Not recorded | 2024-03-15 · model_announcement | LLM Stats |
| Phi-4-multimodal-instruct | 0.55 | Not recorded | Not recorded | 2025-02-01 · model_announcement | LLM Stats |