Original paper v1, submitted 4 November 2026; three L40 deployments and Table 1. Reviewed 5 November; execution dates are unspecified.
Build the server and choose its load
The agent changes batching, scheduling, caching and kernels in an offline Harbor environment. AgentX replay preserves dependent turns, shared prefixes and parallel branches; content is reconstructed while lengths and dependencies remain. The three model/hardware tasks have different prices and latency thresholds.
Value per GPU dollar
Only completed requests meeting both frozen TTFT and TPOT limits contribute token value. GPU allocation and measurement duration form the denominator. Cached input is priced fully once per logical request. Development API spending is separate; this ratio is neither profit nor a percentage.
Verification determines score eligibility
Each agent/deployment gets eight development hours and a 1,800-second final replay. Maximum reasoning is used; GPT uses Codex and other agents Claude Code. The reported strongest verified submission can include one rerun for an anomalous failure. Fresh deployment, disjoint seeds, fidelity checks and human-assisted review precede scoring.
Gemma-4-E4B-it / 1×L40
Original Table 1. A failed verification has no numeric score; the 33 entries include 22 verified submissions. TTFT is in seconds and TPOT in milliseconds.
| Agent | $ per $ | Concurrency | p90 TTFT (s) | p90 TPOT (ms) |
|---|---|---|---|---|
| Claude Opus 5.5 | 2.47 | 40 | 3.01 | 118.59 |
| Claude Sonnet 5.5 | 2.97 | 32 | 2.75 | 83.05 |
| GPT 6 Astra | 1.75 | 32 | 4.77 | 94.19 |
| GPT 6.1 Sol | 1.59 | 24 | 4.43 | 72.45 |
| Claude Opus 5 | 1.15 | 24 | 3.98 | 109.12 |
| GPT 6 Sol | 0.55 | 5 | 3.63 | 110.52 |
| Kimi K3 | 0.52 | 6 | 8.67 | 55.32 |
| Qwen3.8-Max-0902 | 0.87 | 14 | 2.61 | 59.16 |
| MiMo V2.6 Pro | 0.22 | 10 | 43.63 | 108.96 |
| GLM 5.3 | Not passed | — | — | — |
| DeepSeek V4.1 Flash | Not passed | — | — | — |
Not passed retains an unscored verification failure; it is not zero.
Qwen3.5-9B / 2×L40
Original Table 1. A failed verification has no numeric score; the 33 entries include 22 verified submissions. TTFT is in seconds and TPOT in milliseconds.
| Agent | $ per $ | Concurrency | p90 TTFT (s) | p90 TPOT (ms) |
|---|---|---|---|---|
| Claude Opus 5.5 | 4.40 | 16 | 2.25 | 28.16 |
| Claude Sonnet 5.5 | 4.34 | 20 | 12.34 | 26.76 |
| GPT 6 Astra | 3.18 | 12 | 15.62 | 21.94 |
| GPT 6.1 Sol | 2.84 | 4 | 2.71 | 23.67 |
| Claude Opus 5 | 1.16 | 1 | 5.51 | 28.09 |
| GPT 6 Sol | 1.11 | 1 | 13.19 | 20.45 |
| Kimi K3 | 0.93 | 1 | 4.99 | 33.12 |
| Qwen3.8-Max-0902 | Not passed | — | — | — |
| MiMo V2.6 Pro | Not passed | — | — | — |
| GLM 5.3 | Not passed | — | — | — |
| DeepSeek V4.1 Flash | Not passed | — | — | — |
Not passed retains an unscored verification failure; it is not zero.
Qwen3.8-27B-INT4 / 2×L40
Original Table 1. A failed verification has no numeric score; the 33 entries include 22 verified submissions. TTFT is in seconds and TPOT in milliseconds.
| Agent | $ per $ | Concurrency | p90 TTFT (s) | p90 TPOT (ms) |
|---|---|---|---|---|
| Claude Opus 5.5 | 2.64 | 8 | 11.44 | 46.40 |
| Claude Sonnet 5.5 | 2.01 | 12 | 46.57 | 47.35 |
| GPT 6 Astra | 1.48 | 4 | 12.96 | 42.83 |
| GPT 6.1 Sol | 0.98 | 4 | 75.66 | 43.31 |
| Claude Opus 5 | 0.62 | 6 | 33.55 | 62.97 |
| GPT 6 Sol | 0.26 | 2 | 136.48 | 49.53 |
| Kimi K3 | Not passed | — | — | — |
| Qwen3.8-Max-0902 | Not passed | — | — | — |
| MiMo V2.6 Pro | Not passed | — | — | — |
| GLM 5.3 | Not passed | — | — | — |
| DeepSeek V4.1 Flash | Not passed | — | — | — |
Not passed retains an unscored verification failure; it is not zero.
Framework reference systems
References average five independent replay seeds at fixed concurrency. They are framework deployments, separate from coding-agent submissions.
| Deployment | Framework | $ per $ | Concurrency |
|---|---|---|---|
| Gemma E4B / 1×L40 | vLLM v0.28.0 | 0.61 | 8 |
| Gemma E4B / 1×L40 | SGLang v0.5.19 | 0.16 | 2 |
| Qwen3.5 9B / 2×L40 | SGLang v0.5.19 | 1.62 | 1 |
| Qwen3.5 9B / 2×L40 | vLLM v0.28.0 | 1.49 | 1 |
| Qwen3.8 27B INT4 / 2×L40 | vLLM v0.28.0 | 1.47 | 1 |
| Qwen3.8 27B INT4 / 2×L40 | SGLang v0.5.19 | 1.42 | 4 |
Framework references average five seeds; they are not agent-development runs.
Frozen deployment parameters
Changing the checkpoint, hardware, workload or inference policy requires recalibration. These are paper settings, not current commercial price quotes.
| Parameter | Qwen3.5 9B | Gemma E4B | Qwen3.8 27B INT4 |
|---|---|---|---|
| GPU allocation | 2×L40 | 1×L40 | 2×L40 |
| Context limit (tokens) | 1,048,576 | 131,072 | 1,048,576 |
| Input price (USD / million tokens) | 0.1044 | 0.0200 | 0.0898 |
| Output price (USD / million tokens) | 0.1855 | 0.1000 | 2.6570 |
| TTFT target (s) | 4.10028 | 2.52693 | 10.2988 |
| TPOT target (ms) | 23.4974 | 121.7057 | 48.3769 |
| GPU cost (USD / GPU-hour) | 0.82 | 0.82 | 0.82 |
| SGLang reference concurrency | 1 | 2 | 4 |
| vLLM reference concurrency | 1 | 8 | 1 |
Paper-fixed prices and latency targets; hardware order follows original Table 2.
Development costs are a separate measurement
These API cost estimates reprice recorded token and cache usage at OpenRouter rates retrieved on 13 October 2026. They do not replace the allocated GPU denominator of the serving score.
| Agent | Qwen3.5 9B | Gemma E4B |
|---|---|---|
| Claude Opus 5.5 | 99.69 | 104.25 |
| Claude Sonnet 5.5 | 78.60 | 30.72 |
| GPT 6 Astra | 110.81 | 119.79 |
| GPT 6.1 Sol | 13.01 | 13.67 |
| Claude Opus 5 | 108.90 | 104.37 |
| GPT 6 Sol | 50.88 | 35.93 |
| Kimi K3 | 126.98 | 28.71 |
| Qwen3.8-Max-0902 | 46.13 | 43.33 |
| MiMo V2.6 Pro | 6.32 | 10.19 |
| GLM 5.3 | 16.80 | 10.79 |
| DeepSeek V4.1 Flash | 1.64 | 2.07 |
USD estimates use rates retrieved on 13 October 2026, separate from serving GPU cost.
Fixed-load version comparisons
These ratios compare successive implementations on matched replay seeds, rather than adding agent models to the main table.
| System | Versions | Fixed concurrency | Matched-seed ratios |
|---|---|---|---|
| Opus 5.5 / Qwen3.5 9B | v1 → v2 | 16 | 1.88, 1.32 |
| Opus 5 / Qwen3.5 9B | v3 → v4 | 2 | 1.90, 1.67, 1.44 |
| Sonnet 5.5 / Gemma E4B | v3 → v4 | 32 | 1.24, 1.02, 1.05 |
Each listed ratio corresponds to one scoreable matched-seed comparison at the stated fixed load.
Controlled follow-up comparisons
These frozen-server follow-ups use fresh common seeds. Capacity changes Qwen v1 to v6 at concurrency 16; draft length changes Gemma K=3 to K=7 at concurrency 40; replay feedback compares two resumed Opus 5 Gemma servers. D is qualified value, G gross value and f the qualified share. All arms pass verification.
| Comparison | Seed | D before | D after | D ratio | G ratio | f ratio |
|---|---|---|---|---|---|---|
| Capacity | D | 3.533 | 4.761 | 1.348 | 1.110 | 1.214 |
| Capacity | E | 4.294 | 4.960 | 1.155 | 1.074 | 1.076 |
| Draft length | D | 2.240 | 2.308 | 1.030 | 1.030 | 1.000 |
| Draft length | E | 2.450 | 2.544 | 1.039 | 1.029 | 1.009 |
| Replay feedback | F | 0.786 | 0.888 | 1.129 | 1.112 | 1.015 |
| Replay feedback | G | 0.878 | 1.148 | 1.308 | 1.243 | 1.052 |
After/before ratios describe controlled server variants, separate from the main agent comparison.
Read deployment-specific results
The score depends on price accounting, latency targets, offered concurrency and verification. Repeated replay of a frozen implementation differs from independent agent construction. The publication date is known; exact execution dates and full endpoint revisions require the campaign records.