This guide covers the 2024 and 2025 AIME I/II papers. Some linked records also contain 2026 or unspecified-year results; the 30-question descriptions below do not establish the task count or protocol for those rows. Check each result’s year and protocol before comparing.
What the model must produce
AIME is a mathematics competition whose answers are integers from 0 to 999. In the AI evaluation, a model solves a problem and provides that integer; eloquent reasoning alone does not substitute for a correct final answer.
MAA · AIME competition and integer answer format · Artificial Analysis · AIME 2025 evaluation
Keep 2024 and 2025 separate
The commonly used 2024 collection contains 30 problems from AIME I and II. The 2025 collection also contains 30, but different, problems. Equal sizes do not make scores interchangeable. This guide does not cover AIME 2026.
Hugging Face H4 · AIME 2024 dataset · MathArena · AIME 2025 dataset
Answer extraction is part of the test
Evaluators must extract the model's final answer before checking it. Artificial Analysis documents its prompt and mathematical answer-checking procedure. Record these details alongside tool access and reasoning limits, because they define the evaluated setup.
Artificial Analysis · Mathematical answer extraction and evaluation
Distinguish repeated attempts and voting
Ask whether a reported percentage averages attempts, accepts any successful attempt, or grades a voted answer. These answer different questions about reliability and compute. MathArena exposes run configuration and saved outputs so the protocol can be inspected.
Read small differences cautiously
On one 30-question run, one additional correct answer changes accuracy by about 3.33 percentage points. This is arithmetic, not an uncertainty estimate. Inspect per-problem outcomes and repeated runs before treating a small lead as broad mathematical superiority.
Hugging Face H4 · AIME 2024 dataset · MathArena · AIME 2025 dataset