CAD benchmarks: from computer use to engineering verification

Start here: CADWorld and the saved engineering artifact

CADWorld tests agents operating FreeCAD through screenshots and GUI actions. Its 200 tasks cover 11 workflows, from sketches and parts to assemblies, manufacturing and simulation. The main setup allows 100 GUI steps per task; task-specific checks inspect the saved project and required outputs.

Keep the result editions separate: paper v1 reports a best agent success rate of 17.5%; the project site reviewed on November 6, 2026 reports 41.5% after adding extended runs. The 87.0% human result is a single expert reference pass. The separate 50-task terminal experiment is not the full GUI benchmark.

• CADWorldClaire Radar

CADWorld · project, paper and extended evaluation editions · CADWorld · September 2026 paper v1 · CADWorld · official code and evaluator · CADWorld · official task dataset

Four questions to choose a CAD benchmark

Do you need an agent to operate an application, generate an editable program, understand a drawing, or preserve engineering behavior? Use the routes below to choose a task, then open its original publication and evaluation settings.

BenchCAD: executable programs and tool-assisted runs

BenchCAD 1.0 contains 17,900 verified CadQuery programs across 106 industrial part families, with visual/code question answering, image-to-code and code-editing tasks. The two Radar records below retain the collected Vision2Code results with and without a Python tool as separate protocols.

The current author site also presents BenchCAD 2.0 Core, with 100 held-out cases across six tasks. Treat it as a separate edition; a newer website score does not replace an older collected result.

• BenchCADLLM Stats
• BenchCAD (with Python tool)LLM Stats

BenchCAD 1.0 · introducing paper · BenchCAD · current project and versioned results

CADEngBench and MUSE: from a shape to a working design

CADEngBench separates parametric parts from assembly reasoning. It checks validity, engineering requirements, parameter changes, edits, declared linear-static simulation and joints. Read the part, pair and simulation denominators separately. The linked guide includes the paper tables and public-package limitations.

MUSE focuses on multi-component editable B-Rep assemblies. Its October 19, 2026 v3 evaluates execution, geometry, structure/interface fidelity and functional consistency. Structure and Interface Evaluation checks component relationships and interface clearances; this version should be distinguished from older rubric-based descriptions.

• CADEngBenchClaire Radar
• MUSEClaire Radar

CADEngBench · parametric design, assembly and simulation · MUSE · assembly structure and interface geometry, v3

RealCADBench and OmniMech: reading engineering intent

RealCADBench v2 describes 12,632 tasks, but its reported evaluation uses a 1,770-task slice, including 25 assembly tasks. Inputs include text, drawings, photos and renders. Generated FreeCAD API programs are assessed for execution, geometry and visual-semantic identity; these measurements answer different questions.

OmniMech pairs more than 251,000 dimensioned and toleranced drawings with CAD representations. Its tasks cover program synthesis, drawing-to-3D reasoning, annotations and tool-assisted reasoning. The paper announces a data/code release; verify the actual package before planning a reproduction.

• RealCADBenchClaire Radar
• OmniMechClaire Radar

RealCADBench · industrial design intents, v2 · OmniMech · multimodal mechanical benchmark

UniCAD, PNI-3D and segmentation: different kinds of geometry understanding

UniCAD combines point-to-CAD reconstruction, text/image-to-CAD generation and CAD question answering. Evaluate each task with its own output and metric rather than treating the combined suite as one application-control score.

PNI-3D provides 100 parametric CAD families and 2,400 assets across 12 representations. It examines matching, retrieval and robustness to representation changes and sensor distortions. Retrieval accuracy does not establish the ability to create a CAD design.

A Real-World CAD Segmentation Benchmark studies fastener-related hole detection in engineering CAD. It compares rule-based and learned segmentation, including cross-domain behavior. Use it for feature recognition and segmentation, with the linked guide retaining the reported tables.

• UniCADClaire Radar
• PNI-3D Multimodal CAD & Geometric Robustness BenchmarkClaire Radar
• A Real-World CAD Segmentation BenchmarkClaire Radar

UniCAD · multimodal and multitask CAD · PNI-3D · author dataset card and files · A Real-World CAD Segmentation Benchmark · original paper

Three more primary publications to explore

CADBench (MIT DeCoDE Lab) evaluates multimodal CAD program generation using 18,000 samples across six families and five input modalities. Read geometric overlap, surface distance, valid-shape rate and program compactness separately. It is a different project from BenchCAD and Parametric CAD Bench.

CADTestBench uses executable CADTests to check geometric and topological requirements expressed in a text prompt. It complements shape-similarity measures by asking whether the requested conditions actually hold.

Parametric CAD Bench v3 contains 100 FreeCAD tasks: 30 text creation, 30 create-and-edit and 40 engineering-drawing tasks. Its paper was submitted on November 2, 2026, using a September 24 evaluation snapshot. The primary score is mean continuous reward, not binary success rate.

For these three projects, follow the original publications below. This column provides a reading route; their numeric leaderboards have not been imported into Radar as part of this update.

CADBench · MIT DeCoDE Lab project and evaluation · CADTestBench · Text-to-CAD Evaluation with CADTests · Parametric CAD Bench v3 · original November 2026 paper

Before comparing results or running an evaluation

Record the edition, task split, exact model and agent, allowed interface, tool access, retries and action/token/time budget. Keep native CAD files, execution logs and verifier outputs so that a reported result can be inspected.

Task success, continuous reward, IoU, retrieval and segmentation are different measurements. Check whether failures count as zero and whether averages use the full set or only executable outputs. Choose evidence that matches the work you intend to automate.

CADWorld · September 2026 paper v1 · Parametric CAD Bench v3 · original November 2026 paper · CADBench · MIT DeCoDE Lab project and evaluation