AI EVALUATION, IN CONTEXT.BY EQUALITY MACHINE
Benchmark Radar.Benchmarks. Model results. Original sources.
/

Reasoning and math benchmarks

Read reasoning results through the problem type, permitted tools and definition of a correct answer.

Reasoning benchmarks test whether a system can connect information to reach a justified answer. Mathematical tasks can involve computation, algebra or proof-like problem solving; other tasks rely on scientific knowledge or logical constraints. MATH uses competition problems, whereas GPQA asks expert-level science questions. These settings overlap in reasoning demands but do not measure the same knowledge or output.

What these benchmarks test

  • Problem structure matters: selecting an option, calculating an answer and constructing a proof have different failure modes.
  • Knowledge-heavy questions combine recall with reasoning; a wrong answer does not identify which failed.
  • Multi-step tasks can test consistency across intermediate decisions, even when only the final answer is graded.

How to read the results

  • Check the subset, subject mix and whether questions are multiple choice or open response.
  • Match access to calculators, code execution, search and examples in the prompt.
  • Compare results at similar reasoning and sampling budgets; distinguish a single answer from selecting among several attempts.

What to watch for

  • A correct final answer does not automatically validate the explanation or every intermediate step.
  • High performance on a fixed problem collection does not establish general reasoning across unfamiliar domains.

References

  1. Measuring Mathematical Problem Solving With the MATH Dataset
  2. GPQA — A Graduate-Level Google-Proof Q&A Benchmark
  3. Open LLM Leaderboard benchmark documentation

Benchmarks in this topic 674 source records

Search and filter this topic

Related topics

Science benchmarksCoding benchmarksLong context benchmarks