| Claude Opus 5.5 | 76.8 | percent | High reasoning effort; Table 5 overall across eight equally weighted dimensions; only tasks this model solves alone; 240 public+held-out tasks; three test seeds; fixed live partners and scripted teammate; LLM message routing with deterministic outcome checks; paper publication date, exact run date unknown | 2026-11-02 · document_publication | Collaborative Intelligence Benchmark · original v1 evaluation |
| Claude Sonnet 5.5 | 67.9 | percent | High reasoning effort; Table 5 overall across eight equally weighted dimensions; only tasks this model solves alone; 240 public+held-out tasks; three test seeds; fixed live partners and scripted teammate; LLM message routing with deterministic outcome checks; paper publication date, exact run date unknown | 2026-11-02 · document_publication | Collaborative Intelligence Benchmark · original v1 evaluation |
| GPT-6 Astra | 79.4 | percent | High reasoning effort; Table 5 overall across eight equally weighted dimensions; only tasks this model solves alone; 240 public+held-out tasks; three test seeds; fixed live partners and scripted teammate; LLM message routing with deterministic outcome checks; paper publication date, exact run date unknown | 2026-11-02 · document_publication | Collaborative Intelligence Benchmark · original v1 evaluation |
| GPT-6.1 Sol | 78.7 | percent | High reasoning effort; Table 5 overall across eight equally weighted dimensions; only tasks this model solves alone; 240 public+held-out tasks; three test seeds; fixed live partners and scripted teammate; LLM message routing with deterministic outcome checks; paper publication date, exact run date unknown | 2026-11-02 · document_publication | Collaborative Intelligence Benchmark · original v1 evaluation |
| Gemini 4 Argon | 82.2 | percent | High reasoning effort; Table 5 overall across eight equally weighted dimensions; only tasks this model solves alone; 240 public+held-out tasks; three test seeds; fixed live partners and scripted teammate; LLM message routing with deterministic outcome checks; paper publication date, exact run date unknown | 2026-11-02 · document_publication | Collaborative Intelligence Benchmark · original v1 evaluation |
| Kimi K3.1 Pro | 56.1 | percent | High reasoning effort; Table 5 overall across eight equally weighted dimensions; only tasks this model solves alone; 240 public+held-out tasks; three test seeds; fixed live partners and scripted teammate; LLM message routing with deterministic outcome checks; paper publication date, exact run date unknown | 2026-11-02 · document_publication | Collaborative Intelligence Benchmark · original v1 evaluation |
| Claude Fable 5.1 | 65.3 | percent | Lightest reasoning setting offered by the model API; Table 5 overall across eight equally weighted dimensions; only tasks this model solves alone; 240 public+held-out tasks; three test seeds; fixed live partners and scripted teammate; LLM message routing with deterministic outcome checks; paper publication date, exact run date unknown | 2026-11-02 · document_publication | Collaborative Intelligence Benchmark · original v1 evaluation |
| Claude Haiku 5.5 | 51.7 | percent | Lightest reasoning setting offered by the model API; Table 5 overall across eight equally weighted dimensions; only tasks this model solves alone; 240 public+held-out tasks; three test seeds; fixed live partners and scripted teammate; LLM message routing with deterministic outcome checks; paper publication date, exact run date unknown | 2026-11-02 · document_publication | Collaborative Intelligence Benchmark · original v1 evaluation |
| Claude Opus 5.5 | 71.0 | percent | Lightest reasoning setting offered by the model API; Table 5 overall across eight equally weighted dimensions; only tasks this model solves alone; 240 public+held-out tasks; three test seeds; fixed live partners and scripted teammate; LLM message routing with deterministic outcome checks; paper publication date, exact run date unknown | 2026-11-02 · document_publication | Collaborative Intelligence Benchmark · original v1 evaluation |
| Claude Sonnet 5.5 | 65.2 | percent | Lightest reasoning setting offered by the model API; Table 5 overall across eight equally weighted dimensions; only tasks this model solves alone; 240 public+held-out tasks; three test seeds; fixed live partners and scripted teammate; LLM message routing with deterministic outcome checks; paper publication date, exact run date unknown | 2026-11-02 · document_publication | Collaborative Intelligence Benchmark · original v1 evaluation |
| DeepSeek V4 Pro | 39.4 | percent | Lightest reasoning setting offered by the model API; Table 5 overall across eight equally weighted dimensions; only tasks this model solves alone; 240 public+held-out tasks; three test seeds; fixed live partners and scripted teammate; LLM message routing with deterministic outcome checks; paper publication date, exact run date unknown | 2026-11-02 · document_publication | Collaborative Intelligence Benchmark · original v1 evaluation |
| DeepSeek V4.1 Flash | 50.8 | percent | Lightest reasoning setting offered by the model API; Table 5 overall across eight equally weighted dimensions; only tasks this model solves alone; 240 public+held-out tasks; three test seeds; fixed live partners and scripted teammate; LLM message routing with deterministic outcome checks; paper publication date, exact run date unknown | 2026-11-02 · document_publication | Collaborative Intelligence Benchmark · original v1 evaluation |
| GLM-5.3 | 45.8 | percent | Lightest reasoning setting offered by the model API; Table 5 overall across eight equally weighted dimensions; only tasks this model solves alone; 240 public+held-out tasks; three test seeds; fixed live partners and scripted teammate; LLM message routing with deterministic outcome checks; paper publication date, exact run date unknown | 2026-11-02 · document_publication | Collaborative Intelligence Benchmark · original v1 evaluation |
| GPT-6 Astra | 76.5 | percent | Lightest reasoning setting offered by the model API; Table 5 overall across eight equally weighted dimensions; only tasks this model solves alone; 240 public+held-out tasks; three test seeds; fixed live partners and scripted teammate; LLM message routing with deterministic outcome checks; paper publication date, exact run date unknown | 2026-11-02 · document_publication | Collaborative Intelligence Benchmark · original v1 evaluation |
| GPT-6 Luna | 43.4 | percent | Lightest reasoning setting offered by the model API; Table 5 overall across eight equally weighted dimensions; only tasks this model solves alone; 240 public+held-out tasks; three test seeds; fixed live partners and scripted teammate; LLM message routing with deterministic outcome checks; paper publication date, exact run date unknown | 2026-11-02 · document_publication | Collaborative Intelligence Benchmark · original v1 evaluation |
| GPT-6.1 Sol | 74.5 | percent | Lightest reasoning setting offered by the model API; Table 5 overall across eight equally weighted dimensions; only tasks this model solves alone; 240 public+held-out tasks; three test seeds; fixed live partners and scripted teammate; LLM message routing with deterministic outcome checks; paper publication date, exact run date unknown | 2026-11-02 · document_publication | Collaborative Intelligence Benchmark · original v1 evaluation |
| Gemini 3.8 Flash | 51.5 | percent | Lightest reasoning setting offered by the model API; Table 5 overall across eight equally weighted dimensions; only tasks this model solves alone; 240 public+held-out tasks; three test seeds; fixed live partners and scripted teammate; LLM message routing with deterministic outcome checks; paper publication date, exact run date unknown | 2026-11-02 · document_publication | Collaborative Intelligence Benchmark · original v1 evaluation |
| Gemini 4 Argon | 76.6 | percent | Lightest reasoning setting offered by the model API; Table 5 overall across eight equally weighted dimensions; only tasks this model solves alone; 240 public+held-out tasks; three test seeds; fixed live partners and scripted teammate; LLM message routing with deterministic outcome checks; paper publication date, exact run date unknown | 2026-11-02 · document_publication | Collaborative Intelligence Benchmark · original v1 evaluation |
| Grok 4.7 | 55.6 | percent | Lightest reasoning setting offered by the model API; Table 5 overall across eight equally weighted dimensions; only tasks this model solves alone; 240 public+held-out tasks; three test seeds; fixed live partners and scripted teammate; LLM message routing with deterministic outcome checks; paper publication date, exact run date unknown | 2026-11-02 · document_publication | Collaborative Intelligence Benchmark · original v1 evaluation |
| Kimi K3 | 45.9 | percent | Lightest reasoning setting offered by the model API; Table 5 overall across eight equally weighted dimensions; only tasks this model solves alone; 240 public+held-out tasks; three test seeds; fixed live partners and scripted teammate; LLM message routing with deterministic outcome checks; paper publication date, exact run date unknown | 2026-11-02 · document_publication | Collaborative Intelligence Benchmark · original v1 evaluation |
| Kimi K3.1 Pro | 59.9 | percent | Lightest reasoning setting offered by the model API; Table 5 overall across eight equally weighted dimensions; only tasks this model solves alone; 240 public+held-out tasks; three test seeds; fixed live partners and scripted teammate; LLM message routing with deterministic outcome checks; paper publication date, exact run date unknown | 2026-11-02 · document_publication | Collaborative Intelligence Benchmark · original v1 evaluation |
| Mistral Large 4 | 44.5 | percent | Lightest reasoning setting offered by the model API; Table 5 overall across eight equally weighted dimensions; only tasks this model solves alone; 240 public+held-out tasks; three test seeds; fixed live partners and scripted teammate; LLM message routing with deterministic outcome checks; paper publication date, exact run date unknown | 2026-11-02 · document_publication | Collaborative Intelligence Benchmark · original v1 evaluation |
| Nemotron 3 Ultra | 30.6 | percent | Lightest reasoning setting offered by the model API; Table 5 overall across eight equally weighted dimensions; only tasks this model solves alone; 240 public+held-out tasks; three test seeds; fixed live partners and scripted teammate; LLM message routing with deterministic outcome checks; paper publication date, exact run date unknown | 2026-11-02 · document_publication | Collaborative Intelligence Benchmark · original v1 evaluation |
| gpt-oss-120b | 19.1 | percent | Lightest reasoning setting offered by the model API; Table 5 overall across eight equally weighted dimensions; only tasks this model solves alone; 240 public+held-out tasks; three test seeds; fixed live partners and scripted teammate; LLM message routing with deterministic outcome checks; paper publication date, exact run date unknown | 2026-11-02 · document_publication | Collaborative Intelligence Benchmark · original v1 evaluation |