AI EVALUATION, IN CONTEXT.BY EQUALITY MACHINE
Benchmark Radar.Benchmarks. Model results. Original sources.
/

Model reports

PPTBench (Einsia)

A coding agent turns a diagram image into one editable PowerPoint slide while preserving its objects, text and process connections.

Official projectOriginal paperOfficial repositoryOfficial projectCatalog source

AI Benchmark Radar / Equality Machine · Reviewed 2026-10-27

Einsia / Navers Lab’s 500-task scientific flow-diagram reconstruction benchmark, using the September 2026 paper v2. A different, earlier benchmark also uses the name PPTBench.

From pixels to editable objects

Each of 500 frozen tasks provides a reference figure from a real arXiv paper and approved raster resources. The agent reconstructs the slide with native text, shapes and connectors; copying the complete reference image into a slide is not a valid reconstruction.

PPTBench paper v2 · PPTBench official repository and reproduction guide

Validity and hard gates come first

A deterministic check verifies the single-slide editable artifact. Three GPT-5.6 Luna High rounds then inspect process semantics and rendering. A semantic or rendering failure in at least two rounds makes the score zero. Otherwise, merged findings drive fixed deductions in layout (30), typography (40) and local graphics (30).

PPTBench paper v2 · PPTBench official repository and reproduction guide

Keep model, harness and effort together

The mean score includes invalid and gated outputs across all 500 tasks. It is not a success percentage. Gate-pass rate and mean detail score conditional on passing answer different questions. The published 36 configurations use Codex, Claude Code or OpenCode, so a row measures the complete model–harness–effort setup.

PPTBench paper v2 · PPTBench official leaderboard

What the score leaves unresolved

The full evaluation includes tasks used to develop the judging protocol and validate judges. Model-based judgments retain uncertainty, and human validation of gates is not validation of every continuous detail score. More inspection correlates with higher scores in the study, but that association does not establish a causal benefit.

PPTBench paper v2

Reproduce the frozen artifact protocol

The repository supplies task manifests, hashes, prompts, score data and a materialization workflow. Official rendering uses LibreOffice 25.8. Code is MIT-licensed and benchmark-authored metadata is CC BY 4.0; figures retain their original paper licenses, so the code license does not cover every source image.

PPTBench official repository and reproduction guide

Original publication

Einsia / Navers Lab’s 500-task scientific flow-diagram reconstruction benchmark, using the September 2026 paper v2. A different, earlier benchmark also uses the name PPTBench.

Reviewed 2026-10-27

Evidence available for this benchmark

This source record contains 36 numeric observations. The source distinguishes 10 model identities or configurations. A model may appear more than once; check the version, tools and run budget before comparing. Read results and row-level evidence →

Reported metric
Mean gated reconstruction score (0–100)
Recorded unit
points
Metric direction
higher is better

Start with these resources

Cited documents and dates (1)

Evaluation context and evidence

Source
Model reports
Publisher
Not recorded
Benchmark release
2026-08-31
Collected
Not recorded
Modality
Not recorded

These are source-reported observations, not independent reruns by this site. Check the task version, evaluation protocol, unit and reporting date before comparing models. Results from different setups are not a universal ranking.

Reported results 36

Shown in source observation order, not ranked. Dates retain the precision provided by the source.

ModelValueUnitProtocolReported date and basisSource
Claude Opus 533.12pointsNative editable single-slide PPTX; 500 tasks; Claude Code; reasoning effort high; artifact validation; GPT-5.6 Luna High judge, 3 rounds; majority semantic/render gate; deterministic 30/40/30 detail deductions; mean over all tasks including gated zeros2026-09-28 · document_publicationPPTBench paper v2 · Table 1
Claude Sonnet 58.76pointsNative editable single-slide PPTX; 500 tasks; Claude Code; reasoning effort high; artifact validation; GPT-5.6 Luna High judge, 3 rounds; majority semantic/render gate; deterministic 30/40/30 detail deductions; mean over all tasks including gated zeros2026-09-28 · document_publicationPPTBench paper v2 · Table 1
Claude Opus 517.9pointsNative editable single-slide PPTX; 500 tasks; Claude Code; reasoning effort low; artifact validation; GPT-5.6 Luna High judge, 3 rounds; majority semantic/render gate; deterministic 30/40/30 detail deductions; mean over all tasks including gated zeros2026-09-28 · document_publicationPPTBench paper v2 · Table 1
Claude Sonnet 54.03pointsNative editable single-slide PPTX; 500 tasks; Claude Code; reasoning effort low; artifact validation; GPT-5.6 Luna High judge, 3 rounds; majority semantic/render gate; deterministic 30/40/30 detail deductions; mean over all tasks including gated zeros2026-09-28 · document_publicationPPTBench paper v2 · Table 1
Claude Opus 533.54pointsNative editable single-slide PPTX; 500 tasks; Claude Code; reasoning effort max; artifact validation; GPT-5.6 Luna High judge, 3 rounds; majority semantic/render gate; deterministic 30/40/30 detail deductions; mean over all tasks including gated zeros2026-09-28 · document_publicationPPTBench paper v2 · Table 1
Claude Opus 524.13pointsNative editable single-slide PPTX; 500 tasks; Claude Code; reasoning effort medium; artifact validation; GPT-5.6 Luna High judge, 3 rounds; majority semantic/render gate; deterministic 30/40/30 detail deductions; mean over all tasks including gated zeros2026-09-28 · document_publicationPPTBench paper v2 · Table 1
Claude Sonnet 55.99pointsNative editable single-slide PPTX; 500 tasks; Claude Code; reasoning effort medium; artifact validation; GPT-5.6 Luna High judge, 3 rounds; majority semantic/render gate; deterministic 30/40/30 detail deductions; mean over all tasks including gated zeros2026-09-28 · document_publicationPPTBench paper v2 · Table 1
Claude Opus 535.91pointsNative editable single-slide PPTX; 500 tasks; Claude Code; reasoning effort xhigh; artifact validation; GPT-5.6 Luna High judge, 3 rounds; majority semantic/render gate; deterministic 30/40/30 detail deductions; mean over all tasks including gated zeros2026-09-28 · document_publicationPPTBench paper v2 · Table 1
Claude Sonnet 58.98pointsNative editable single-slide PPTX; 500 tasks; Claude Code; reasoning effort xhigh; artifact validation; GPT-5.6 Luna High judge, 3 rounds; majority semantic/render gate; deterministic 30/40/30 detail deductions; mean over all tasks including gated zeros2026-09-28 · document_publicationPPTBench paper v2 · Table 1
Qwen 3.8 Max47.69pointsNative editable single-slide PPTX; 500 tasks; Claude Code; reasoning effort xhigh; artifact validation; GPT-5.6 Luna High judge, 3 rounds; majority semantic/render gate; deterministic 30/40/30 detail deductions; mean over all tasks including gated zeros2026-09-28 · document_publicationPPTBench paper v2 · Table 1
GPT-5.6 Luna19.47pointsNative editable single-slide PPTX; 500 tasks; Codex; reasoning effort high; artifact validation; GPT-5.6 Luna High judge, 3 rounds; majority semantic/render gate; deterministic 30/40/30 detail deductions; mean over all tasks including gated zeros2026-09-28 · document_publicationPPTBench paper v2 · Table 1
GPT-5.6 Sol28.28pointsNative editable single-slide PPTX; 500 tasks; Codex; reasoning effort high; artifact validation; GPT-5.6 Luna High judge, 3 rounds; majority semantic/render gate; deterministic 30/40/30 detail deductions; mean over all tasks including gated zeros2026-09-28 · document_publicationPPTBench paper v2 · Table 1
GPT-5.6 Terra16.89pointsNative editable single-slide PPTX; 500 tasks; Codex; reasoning effort high; artifact validation; GPT-5.6 Luna High judge, 3 rounds; majority semantic/render gate; deterministic 30/40/30 detail deductions; mean over all tasks including gated zeros2026-09-28 · document_publicationPPTBench paper v2 · Table 1
GPT-6-Astra77.34pointsNative editable single-slide PPTX; 500 tasks; Codex; reasoning effort high; artifact validation; GPT-5.6 Luna High judge, 3 rounds; majority semantic/render gate; deterministic 30/40/30 detail deductions; mean over all tasks including gated zeros2026-09-28 · document_publicationPPTBench paper v2 · Table 1
GPT-5.6 Luna3.03pointsNative editable single-slide PPTX; 500 tasks; Codex; reasoning effort low; artifact validation; GPT-5.6 Luna High judge, 3 rounds; majority semantic/render gate; deterministic 30/40/30 detail deductions; mean over all tasks including gated zeros2026-09-28 · document_publicationPPTBench paper v2 · Table 1
GPT-5.6 Sol11.12pointsNative editable single-slide PPTX; 500 tasks; Codex; reasoning effort low; artifact validation; GPT-5.6 Luna High judge, 3 rounds; majority semantic/render gate; deterministic 30/40/30 detail deductions; mean over all tasks including gated zeros2026-09-28 · document_publicationPPTBench paper v2 · Table 1
GPT-5.6 Terra8.6pointsNative editable single-slide PPTX; 500 tasks; Codex; reasoning effort low; artifact validation; GPT-5.6 Luna High judge, 3 rounds; majority semantic/render gate; deterministic 30/40/30 detail deductions; mean over all tasks including gated zeros2026-09-28 · document_publicationPPTBench paper v2 · Table 1
GPT-6-Astra59.42pointsNative editable single-slide PPTX; 500 tasks; Codex; reasoning effort low; artifact validation; GPT-5.6 Luna High judge, 3 rounds; majority semantic/render gate; deterministic 30/40/30 detail deductions; mean over all tasks including gated zeros2026-09-28 · document_publicationPPTBench paper v2 · Table 1
GPT-5.6 Luna31.02pointsNative editable single-slide PPTX; 500 tasks; Codex; reasoning effort max; artifact validation; GPT-5.6 Luna High judge, 3 rounds; majority semantic/render gate; deterministic 30/40/30 detail deductions; mean over all tasks including gated zeros2026-09-28 · document_publicationPPTBench paper v2 · Table 1
GPT-5.6 Sol49.28pointsNative editable single-slide PPTX; 500 tasks; Codex; reasoning effort max; artifact validation; GPT-5.6 Luna High judge, 3 rounds; majority semantic/render gate; deterministic 30/40/30 detail deductions; mean over all tasks including gated zeros2026-09-28 · document_publicationPPTBench paper v2 · Table 1
GPT-5.6 Terra33.46pointsNative editable single-slide PPTX; 500 tasks; Codex; reasoning effort max; artifact validation; GPT-5.6 Luna High judge, 3 rounds; majority semantic/render gate; deterministic 30/40/30 detail deductions; mean over all tasks including gated zeros2026-09-28 · document_publicationPPTBench paper v2 · Table 1
GPT-6-Astra68.76pointsNative editable single-slide PPTX; 500 tasks; Codex; reasoning effort max; artifact validation; GPT-5.6 Luna High judge, 3 rounds; majority semantic/render gate; deterministic 30/40/30 detail deductions; mean over all tasks including gated zeros2026-09-28 · document_publicationPPTBench paper v2 · Table 1
GPT-5.59.64pointsNative editable single-slide PPTX; 500 tasks; Codex; reasoning effort medium; artifact validation; GPT-5.6 Luna High judge, 3 rounds; majority semantic/render gate; deterministic 30/40/30 detail deductions; mean over all tasks including gated zeros2026-09-28 · document_publicationPPTBench paper v2 · Table 1
GPT-5.6 Luna8.15pointsNative editable single-slide PPTX; 500 tasks; Codex; reasoning effort medium; artifact validation; GPT-5.6 Luna High judge, 3 rounds; majority semantic/render gate; deterministic 30/40/30 detail deductions; mean over all tasks including gated zeros2026-09-28 · document_publicationPPTBench paper v2 · Table 1
GPT-5.6 Sol22.2pointsNative editable single-slide PPTX; 500 tasks; Codex; reasoning effort medium; artifact validation; GPT-5.6 Luna High judge, 3 rounds; majority semantic/render gate; deterministic 30/40/30 detail deductions; mean over all tasks including gated zeros2026-09-28 · document_publicationPPTBench paper v2 · Table 1
GPT-5.6 Terra10.1pointsNative editable single-slide PPTX; 500 tasks; Codex; reasoning effort medium; artifact validation; GPT-5.6 Luna High judge, 3 rounds; majority semantic/render gate; deterministic 30/40/30 detail deductions; mean over all tasks including gated zeros2026-09-28 · document_publicationPPTBench paper v2 · Table 1
GPT-6-Astra68.16pointsNative editable single-slide PPTX; 500 tasks; Codex; reasoning effort medium; artifact validation; GPT-5.6 Luna High judge, 3 rounds; majority semantic/render gate; deterministic 30/40/30 detail deductions; mean over all tasks including gated zeros2026-09-28 · document_publicationPPTBench paper v2 · Table 1
GPT-5.6 Luna1.53pointsNative editable single-slide PPTX; 500 tasks; Codex; reasoning effort none; artifact validation; GPT-5.6 Luna High judge, 3 rounds; majority semantic/render gate; deterministic 30/40/30 detail deductions; mean over all tasks including gated zeros2026-09-28 · document_publicationPPTBench paper v2 · Table 1
GPT-5.6 Sol4.99pointsNative editable single-slide PPTX; 500 tasks; Codex; reasoning effort none; artifact validation; GPT-5.6 Luna High judge, 3 rounds; majority semantic/render gate; deterministic 30/40/30 detail deductions; mean over all tasks including gated zeros2026-09-28 · document_publicationPPTBench paper v2 · Table 1
GPT-5.6 Terra3.55pointsNative editable single-slide PPTX; 500 tasks; Codex; reasoning effort none; artifact validation; GPT-5.6 Luna High judge, 3 rounds; majority semantic/render gate; deterministic 30/40/30 detail deductions; mean over all tasks including gated zeros2026-09-28 · document_publicationPPTBench paper v2 · Table 1
GPT-5.6 Luna24.63pointsNative editable single-slide PPTX; 500 tasks; Codex; reasoning effort xhigh; artifact validation; GPT-5.6 Luna High judge, 3 rounds; majority semantic/render gate; deterministic 30/40/30 detail deductions; mean over all tasks including gated zeros2026-09-28 · document_publicationPPTBench paper v2 · Table 1
GPT-5.6 Sol36.24pointsNative editable single-slide PPTX; 500 tasks; Codex; reasoning effort xhigh; artifact validation; GPT-5.6 Luna High judge, 3 rounds; majority semantic/render gate; deterministic 30/40/30 detail deductions; mean over all tasks including gated zeros2026-09-28 · document_publicationPPTBench paper v2 · Table 1
GPT-5.6 Terra25.84pointsNative editable single-slide PPTX; 500 tasks; Codex; reasoning effort xhigh; artifact validation; GPT-5.6 Luna High judge, 3 rounds; majority semantic/render gate; deterministic 30/40/30 detail deductions; mean over all tasks including gated zeros2026-09-28 · document_publicationPPTBench paper v2 · Table 1
GPT-6-Astra72.6pointsNative editable single-slide PPTX; 500 tasks; Codex; reasoning effort xhigh; artifact validation; GPT-5.6 Luna High judge, 3 rounds; majority semantic/render gate; deterministic 30/40/30 detail deductions; mean over all tasks including gated zeros2026-09-28 · document_publicationPPTBench paper v2 · Table 1
Kimi K367.8pointsNative editable single-slide PPTX; 500 tasks; OpenCode; reasoning effort high; artifact validation; GPT-5.6 Luna High judge, 3 rounds; majority semantic/render gate; deterministic 30/40/30 detail deductions; mean over all tasks including gated zeros2026-09-28 · document_publicationPPTBench paper v2 · Table 1
GLM-5.3-Flash32.42pointsNative editable single-slide PPTX; 500 tasks; OpenCode; reasoning effort max; artifact validation; GPT-5.6 Luna High judge, 3 rounds; majority semantic/render gate; deterministic 30/40/30 detail deductions; mean over all tasks including gated zeros2026-09-28 · document_publicationPPTBench paper v2 · Table 1

Explore related benchmarks

Suggestions based on shared evidence or source categories, not verified equivalents.

ScienceQAShared source categories →JointAVBenchShared source categories →MMVetShared source categories →CharXiv-DShared source categories →AndroidWorld_SRShared source categories →MLVUShared source categories →MM-MT-BenchShared source categories →PMC-VQAShared source categories →

Put this benchmark in context

Coding benchmarksTools and agents benchmarksMultimodal benchmarks

Continue exploring

Search the catalog using this record’s source labels. Search matches are not verified equivalents.

multimodal →Model reports · all records →