AI benchmarks and leaderboards are different things
Learn the difference between a benchmark, an evaluation run and a leaderboard, and use each to answer the right question.
LLM Stats
BIG-Bench Extra Hard
BIG-Bench Extra Hard (BBEH) is a challenging benchmark that replaces each task in BIG-Bench Hard with a novel task that probes similar reasoning capabilities but exhibits significantly increased difficulty. The benchmark contains 23 tasks testing diverse reasoning skills including many-hop reasoning, causal understanding, spatial reasoning, temporal arithmetic, geometric reasoning, linguistic reasoning, logic puzzles, and humor understanding. Designed to address saturation on existing benchmarks where state-of-the-art models achieve near-perfect scores, BBEH shows substantial room for improvement with best models achieving only 9.8-44.8% average accuracy.
Evaluation context and evidence
- Source
- LLM Stats
- Publisher
- Not recorded
- Benchmark release
- Not recorded
- Collected
- 2026-08-17
- Modality
- text
These are source-reported observations, not independent reruns by this site. Check the task version, evaluation protocol, unit and reporting date before comparing models. Results from different setups are not a universal ranking.
Original sources
Reported results 11
Shown in source observation order, not ranked. Dates retain the precision provided by the source.
| Model | Value | Unit | Protocol | Reported date and basis | Source |
|---|---|---|---|---|---|
| Gemma 4 31B | 0.744 | Not recorded | Not recorded | 2026-04-02 · model_announcement | LLM Stats |
| Gemma 4 26B-A4B | 0.648 | Not recorded | Not recorded | 2026-04-02 · model_announcement | LLM Stats |
| Gemma 4 12B | 0.53 | Not recorded | Not recorded | 2026-05-23 · model_announcement | LLM Stats |
| DiffusionGemma 26B-A4B | 0.476 | Not recorded | Not recorded | 2026-06-10 · model_announcement | LLM Stats |
| Gemma 4 E4B | 0.331 | Not recorded | Not recorded | 2026-04-02 · model_announcement | LLM Stats |
| Gemma 4 E2B | 0.219 | Not recorded | Not recorded | 2026-04-02 · model_announcement | LLM Stats |
| Gemma 3 27B | 0.193 | Not recorded | Not recorded | 2025-03-12 · model_announcement | LLM Stats |
| Gemma 3 12B | 0.163 | Not recorded | Not recorded | 2025-03-12 · model_announcement | LLM Stats |
| Gemini Diffusion | 0.15 | Not recorded | Not recorded | 2025-05-20 · model_announcement | LLM Stats |
| Gemma 3 4B | 0.11 | Not recorded | Not recorded | 2025-03-12 · model_announcement | LLM Stats |
| Gemma 3 1B | 0.072 | Not recorded | Not recorded | 2025-03-12 · model_announcement | LLM Stats |