The 223-task benchmark and publisher runs in the October 14, 2026 paper v1. Main-table values were checked against the official leaderboard on October 25, 2026.
What the agent creates
Agents deliver editable Blender projects and requested media across nine creative domains. The 223 tasks comprise 78 text, 130 image and 15 video tasks; 59 also include specification attachments. The task suite has 2,651 binary rubric criteria.
BlenderWorld paper v1 · Methods, Table 2 and Appendix D · BlenderWorld · Official benchmark and leaderboard
How the scores work
Overall equally combines weighted requirement satisfaction and visual quality, then averages all 223 tasks on a 0–100 scale. Evaluation uses scene inspection, eight adaptive rendered views and eight quality dimensions. Failed evaluations score zero. Full pass counts tasks satisfying every rubric criterion; it is a separate percentage, not the Overall score.
BlenderWorld paper v1 · Methods, Table 2 and Appendix D · BlenderWorld · Pinned evaluation protocol and runtime
Published completion results
The publisher reports these 13 agents on the same task suite. Read average quality alongside Full pass: a strong average does not establish that every requested property was delivered.
| Model | Overall | Text | MM | Rubric | Dimension | Full pass % |
|---|---|---|---|---|---|---|
| GPT-6-Astra | 79.9 | 78.6 | 80.7 | 87.3 | 72.6 | 42.2 |
| Opus-5 | 73.0 | 74.8 | 72.1 | 80.9 | 65.1 | 25.6 |
| GPT-5.6-Sol | 67.7 | 71.7 | 65.5 | 74.7 | 60.7 | 15.7 |
| DeepSeek-V4.1-Flash | 65.7 | 66.0 | 65.5 | 72.0 | 59.4 | 11.7 |
| Qwen3.8-Max | 60.6 | 59.6 | 61.1 | 65.9 | 55.3 | 10.3 |
| GLM-5.3-Flash | 59.7 | 63.1 | 57.8 | 64.8 | 54.4 | 6.28 |
| Grok-4.6 | 57.0 | 60.0 | 55.4 | 61.4 | 52.7 | 6.73 |
| Opus-4.8 | 57.0 | 58.6 | 56.1 | 60.2 | 53.7 | 5.38 |
| Seed2.1pro-0915 | 51.3 | 54.7 | 49.5 | 54.9 | 47.7 | 3.59 |
| Qwen3.8-27B | 47.7 | 46.1 | 48.6 | 50.0 | 45.5 | 1.35 |
| Kimi-K3 | 46.5 | 41.3 | 49.4 | 47.9 | 45.2 | 1.79 |
| Gemini-3.8-Flash | 34.2 | 35.9 | 33.2 | 29.8 | 38.5 | 0.45 |
| MiniMax-M3 | 31.8 | 34.1 | 30.6 | 28.9 | 34.7 | 0.45 |
Overall, Text, MM, Rubric and Dimension are scores on a 0–100 scale. Full pass is the percentage of tasks passing all rubric criteria. Values and displayed precision follow the paper; failed evaluations remain in the denominator.
BlenderWorld paper v1 · Methods, Table 2 and Appendix D · BlenderWorld · Official benchmark and leaderboard
Rollout resources
These are per-task means across the full suite, including unsuccessful calls. They describe generated output and requested operations; they are not matched compute budgets or monetary costs.
| Model | Turns | Output tokens K | Tool calls |
|---|---|---|---|
| GPT-6-Astra | 45.6 | 84.5 | 79.1 |
| Opus-5 | 158 | 241 | 155 |
| GPT-5.6-Sol | 44.9 | 39.9 | 44.0 |
| DeepSeek-V4.1-Flash | 171 | 209 | 168 |
| Qwen3.8-Max | 75.1 | 90.6 | 72.8 |
| GLM-5.3-Flash | 102 | 145 | 101 |
| Grok-4.6 | 93.4 | 120 | 89.8 |
| Opus-4.8 | 97.7 | 164 | 95.6 |
| Seed2.1pro-0915 | 196 | 122 | 194 |
| Qwen3.8-27B | 90.0 | 112 | 87.4 |
| Kimi-K3 | 60.5 | 54.3 | 58.3 |
| Gemini-3.8-Flash | 56.6 | 35.9 | 54.5 |
| MiniMax-M3 | 132 | 72.2 | 127 |
K means thousands of output tokens. A turn is one model-response round. Different harnesses and model accounting limit direct efficiency comparisons.
BlenderWorld paper v1 · Methods, Table 2 and Appendix D · BlenderWorld · Official benchmark and leaderboard
What comparisons can establish
Each model runs once per task at its highest available reasoning setting. GPT-6-Astra and GPT-5.6-Sol use Codex; the other agents use Claude Code. GPT-5.6-terra performs automatic evaluation. Rankings therefore compare complete agent setups, with no repeated-run variance estimate; they do not isolate base-model ability or certify production readiness.
Before reproducing a result
Pin the task revision, Blender 4.2.10, harness, reasoning configuration and judge. The reference runtime uses Codex CLI 0.147.0, Claude Code 2.1.220 and Node.js 22.17.0; the paper notes that a 48-task MiniMax-M3 batch uses Claude Code 2.1.241. The public evaluator accepts submission ZIPs from other harnesses, so changed setups need separate protocol labels.
BlenderWorld paper v1 · Methods, Table 2 and Appendix D · BlenderWorld · Pinned evaluation protocol and runtime