November 2, 2026 · Paper

Parametric CAD Bench V3: Paper and Evaluation Findings

Parametric CAD Bench v3: Evaluating Agentic Parametric CAD Authoring from Text and Engineering Drawings is now available on arXiv. The paper documents the benchmark methodology, evaluation framework, and findings from a September 24, 2026 snapshot of 13 model-agent systems.

Mei Chen, Yu Xia, Haipeng Mao

Evaluating parametric CAD authoring

Reliable AI for engineering requires reproducible evaluation of the resulting CAD artifacts, including their geometry, specified dimensions, and native modeling history. Executable code and visual similarity alone provide an incomplete account of whether a design request has been fulfilled. Parametric design intent, preservation of modeling history, and feature-level editability are central concerns in practical CAD workflows.

Developed at gNucleus AI, Parametric CAD Bench v3 evaluates agents in a pinned FreeCAD environment on 100 tasks inspired by engineering workflows:

• Text-to-CAD Creation (30 tasks). Construct parametric CAD models from natural-language specifications.
• Text-to-CAD Create-and-Edit (30 tasks). Create a model and modify it according to subsequent textual instructions.
• Image-to-CAD (40 tasks). Reconstruct editable 3D CAD models from engineering drawings.

Agents submit native CAD artifacts to a separate programmatic verifier. The primary metric is mean continuous task reward, combining geometry similarity and specification consistency. Scores therefore reflect graded performance across tasks rather than a binary task-success rate.

Findings from the September 24 snapshot

The paper covers 13 model-agent systems, including systems using Claude, GPT, Gemini, and Grok models. Three findings stand out:

• Claude Code with Claude Opus 5.5 records the highest overall point estimate: 61.03% mean reward across the benchmark and 84.66% on Image-to-CAD.
• Drawing reconstruction performance varies substantially across systems. The Image-to-CAD results show meaningful differences in the ability to translate engineering drawings into native CAD artifacts.
• Agent harnesses affect performance, while complex editing remains challenging. Results depend on the full system configuration, including tools and execution environments, and should not be interpreted as measurements of model capability alone.

Paper and public resources

The benchmark, leaderboard, evaluation configurations, and execution trajectories were publicly released in September 2026. This announcement marks the availability of the paper, which consolidates the methodology and findings from that release. The paper's results describe a dated snapshot; the live leaderboard provides the current public results.