AI EVALUATION, IN CONTEXT.BY EQUALITY MACHINE
Benchmark Radar.Benchmarks. Model results. Original sources.
/

Model reports

Collaborative Intelligence Benchmark

Check how a capable agent behaves when teamwork becomes difficult: withholding facts, pressure, conflict or unfinished work. Scores depend on which tasks it first solves alone.

Original paperOfficial repositoryOfficial projectCatalog source

AI Benchmark Radar / Equality Machine · Reviewed 2026-11-04

Original paper v1, 2 November 2026; reviewed 4 November. All 24 overall result cards, with split scores, dimension profiles and uncertainty separately retained.

Cooperate under pressure

The 240 tasks cover eight teamwork dimensions; 160 are public and 80 held out. Teams have two to five members, including a scripted problematic teammate.

Collaborative Intelligence Benchmark · original paper v1

Read the denominator first

Each model is scored only on tasks it solves alone. Tests repeat three times; conflict permits partial credit. Overall averages the eight dimensions equally, rather than pooling all attempts.

Collaborative Intelligence Benchmark · original paper v1

Reasoning and judges matter

Default means the lightest API reasoning setting. Six models also run at high effort. Message judges control the script; deterministic checks grade outcomes. Execution dates are unspecified.

Collaborative Intelligence Benchmark · original paper v1

Interpret within this task design

Solo-eligible sets differ between models. English scripted scenarios and fixed partners limit generalization; this is not a direct estimate of workplace or human-team performance.

Collaborative Intelligence Benchmark · original paper v1

Public and held-out results

Default settings; intervals and all model-specific eligible counts are retained.

Default-setting checks and splits
ModelOverall %Solved alone / 240Calm-team baseline %Drop (points)Public %Held-out %
Gemini 4 Argon76.6 ± 3.523096.8 ± 1.920.2 ± 4.078.7 ± 4.072.7 ± 6.4
GPT-6 Astra76.5 ± 3.523897.9 ± 1.721.3 ± 3.876.9 ± 4.375.8 ± 5.4
GPT-6.1 Sol74.5 ± 3.523696.8 ± 2.222.3 ± 4.075.9 ± 4.171.7 ± 6.7
Claude Opus 5.571.0 ± 4.022795.9 ± 2.324.9 ± 4.370.6 ± 4.972.9 ± 5.8
Claude Fable 5.165.3 ± 4.418696.7 ± 2.231.5 ± 5.165.2 ± 5.666.4 ± 6.3
Claude Sonnet 5.565.2 ± 3.823192.3 ± 3.227.2 ± 4.464.7 ± 4.866.0 ± 5.9
Grok 4.755.6 ± 3.922094.6 ± 2.739.1 ± 4.555.8 ± 5.055.6 ± 6.4
Claude Haiku 5.551.7 ± 3.721284.3 ± 4.332.6 ± 5.353.5 ± 4.248.7 ± 6.6
Gemini 3.8 Flash51.5 ± 4.220791.5 ± 3.439.9 ± 4.953.0 ± 5.148.0 ± 6.3
Mistral Large 444.5 ± 4.318981.9 ± 5.137.4 ± 6.643.2 ± 5.349.2 ± 7.2
GPT-6 Luna43.4 ± 4.018085.7 ± 4.642.3 ± 5.643.1 ± 4.445.0 ± 7.5
Kimi K3.1 Pro59.9 ± 4.120188.1 ± 4.228.2 ± 5.359.9 ± 5.060.6 ± 7.2
DeepSeek V4.1 Flash50.8 ± 4.321083.3 ± 4.532.6 ± 5.949.7 ± 5.352.8 ± 8.1
Kimi K345.9 ± 3.821286.1 ± 4.240.2 ± 4.644.8 ± 4.548.1 ± 6.4
GLM-5.345.8 ± 4.021688.9 ± 4.043.1 ± 5.347.0 ± 4.943.2 ± 6.9
DeepSeek V4 Pro39.4 ± 4.420286.1 ± 4.246.6 ± 5.538.1 ± 5.742.1 ± 7.3
Nemotron 3 Ultra30.6 ± 4.416672.1 ± 6.241.4 ± 6.629.5 ± 5.032.9 ± 8.7
gpt-oss-120b19.1 ± 5.012964.6 ± 6.045.5 ± 6.119.9 ± 5.918.7 ± 7.0

± is half the 95% bootstrap interval.

Original results · Table 2

All teamwork profiles

Default and high effort remain separate rows.

24 configurations across eight dimensions
Rank in sourceModel / effortUnsafe pressureReportingInformationEmotionKeep on trackConflictInitiativeTakeoverOverall
1Gemini 4 Argon HIGH99 ± 299 ± 297 ± 471 ± 1480 ± 1318 ± 1198 ± 396 ± 582.2 ± 2.9
2GPT-6 Astra HIGH97 ± 5100 ± 092 ± 851 ± 1692 ± 713 ± 897 ± 493 ± 879.4 ± 3.0
3GPT-6.1 Sol HIGH98 ± 3100 ± 087 ± 947 ± 1694 ± 716 ± 1198 ± 391 ± 878.7 ± 3.1
4Claude Opus 5.5 HIGH98 ± 3100 ± 095 ± 559 ± 1690 ± 93 ± 390 ± 1079 ± 1676.8 ± 3.3
5Gemini 4 Argon98 ± 3100 ± 094 ± 653 ± 1467 ± 1530 ± 1273 ± 1298 ± 376.6 ± 3.5
6GPT-6 Astra97 ± 5100 ± 091 ± 743 ± 1694 ± 516 ± 1187 ± 1084 ± 1276.5 ± 3.5
7GPT-6.1 Sol98 ± 3100 ± 087 ± 1037 ± 1686 ± 919 ± 1180 ± 1189 ± 974.5 ± 3.5
8Claude Opus 5.593 ± 8100 ± 092 ± 756 ± 1684 ± 118 ± 772 ± 1564 ± 1671.0 ± 4.0
9Claude Sonnet 5.5 HIGH96 ± 4100 ± 086 ± 1231 ± 1688 ± 105 ± 461 ± 1476 ± 1367.9 ± 3.7
10Claude Fable 5.195 ± 5100 ± 074 ± 1437 ± 1685 ± 139 ± 654 ± 1770 ± 1765.3 ± 4.4
11Claude Sonnet 5.598 ± 399 ± 278 ± 1330 ± 1480 ± 1015 ± 855 ± 1567 ± 1165.2 ± 3.8
12Kimi K3.1 Pro OPEN94 ± 698 ± 352 ± 1516 ± 1076 ± 1410 ± 552 ± 1581 ± 1559.9 ± 4.1
13Kimi K3.1 Pro HIGH OPEN94 ± 796 ± 444 ± 1418 ± 1169 ± 139 ± 627 ± 1392 ± 956.1 ± 3.6
14Grok 4.794 ± 6100 ± 024 ± 1213 ± 1060 ± 1545 ± 1323 ± 1386 ± 1255.6 ± 3.9
15Claude Haiku 5.594 ± 586 ± 1067 ± 1316 ± 1264 ± 1419 ± 65 ± 563 ± 1351.7 ± 3.7
16Gemini 3.8 Flash89 ± 993 ± 838 ± 1516 ± 1268 ± 147 ± 521 ± 1280 ± 1451.5 ± 4.2
17DeepSeek V4.1 Flash OPEN86 ± 1087 ± 940 ± 1311 ± 1055 ± 1339 ± 1326 ± 1462 ± 1650.8 ± 4.3
18Kimi K3 OPEN83 ± 1194 ± 535 ± 127 ± 660 ± 1524 ± 816 ± 1149 ± 1345.9 ± 3.8
19GLM-5.3 OPEN81 ± 1293 ± 544 ± 1317 ± 1164 ± 138 ± 76 ± 853 ± 1645.8 ± 4.0
20Mistral Large 489 ± 988 ± 832 ± 1313 ± 944 ± 1526 ± 1314 ± 1150 ± 1844.5 ± 4.3
21GPT-6 Luna94 ± 585 ± 1020 ± 114 ± 667 ± 1714 ± 108 ± 955 ± 1443.4 ± 4.0
22DeepSeek V4 Pro OPEN69 ± 1582 ± 1013 ± 85 ± 648 ± 1527 ± 1014 ± 1057 ± 1739.4 ± 4.4
23Nemotron 3 Ultra OPEN56 ± 1956 ± 1918 ± 86 ± 548 ± 1553 ± 120 ± 09 ± 1230.6 ± 4.4
24gpt-oss-120b OPEN59 ± 2138 ± 299 ± 72 ± 322 ± 1416 ± 70 ± 07 ± 1019.1 ± 5.0

Percent; ± is half the 95% bootstrap interval. OPEN denotes open weights.

Original results · Table 5

Extra reasoning has a cost

Paired reasoning comparisons are preserved as reported.

Reasoning comparison
ModelDefault %High %Delta pointsCost multiplierTasks betterTasks worseSign-test p
Gemini 4 Argon76.682.2+5.63.3 ×41220.02
GPT-6 Astra76.579.4+2.84.3 ×27130.04
GPT-6.1 Sol74.578.7+4.34.4 ×3310< 0.001
Claude Opus 5.571.076.8+5.74.8 ×36170.01
Claude Sonnet 5.565.267.9+2.72.9 ×40290.23
Kimi K3.1 Pro59.956.1- 3.71.0 ×18360.02

Printed deltas may differ from subtracting rounded displayed scores; values are not recalculated.

Original results · Table 6

Conflict scores and deal outcomes

Joint deal points and the best-deal rate are different quantities.

Default-setting conflict outcomes
ModelConflict score %Best deal %Held firm %Gave in %No deal %Joint points %
Gemini 4 Argon30.028468296.3
GPT-6 Astra16.113681095.5
GPT-6.1 Sol18.917479095.7
Claude Opus 5.57.86490294.4
Claude Fable 5.18.65788093.9
Claude Sonnet 5.515.4101179694.0
Kimi K3.1 Pro9.731483493.3
Grok 4.745.041851495.6
Claude Haiku 5.519.3136631692.9
Gemini 3.8 Flash7.55690092.9
DeepSeek V4.1 Flash39.12235432595.4
Kimi K324.083260791.6
GLM-5.38.35788492.1
Mistral Large 426.21229602194.9
GPT-6 Luna14.3101081591.4
DeepSeek V4 Pro27.274053993.8
Nemotron 3 Ultra53.23046241789.8
gpt-oss-120b16.062074190.2

Conflict awards 1 for the best deal, 0.5 for holding firm and 0 for conceding.

Original results · Table 8

Original publication

Original paper v1, 2 November 2026; reviewed 4 November. All 24 overall result cards, with split scores, dimension profiles and uncertainty separately retained.

Reviewed 2026-11-04

Evidence available for this benchmark

This source record contains 24 numeric observations. The source distinguishes 18 model identities or configurations. A model may appear more than once; check the version, tools and run budget before comparing. Read results and row-level evidence →

Reported metric
overall teamwork test score (equal mean of eight dimensions)
Recorded unit
percent
Metric direction
higher is better

Start with these resources

Cited documents and dates (1)

Evaluation context and evidence

Source
Model reports
Publisher
Not recorded
Benchmark release
2026-11-02
Collected
Not recorded
Modality
Not recorded

These are source-reported observations, not independent reruns by this site. Check the task version, evaluation protocol, unit and reporting date before comparing models. Results from different setups are not a universal ranking.

Reported results 24

Shown in source observation order, not ranked. Dates retain the precision provided by the source.

ModelValueUnitProtocolReported date and basisSource
Claude Opus 5.576.8percentHigh reasoning effort; Table 5 overall across eight equally weighted dimensions; only tasks this model solves alone; 240 public+held-out tasks; three test seeds; fixed live partners and scripted teammate; LLM message routing with deterministic outcome checks; paper publication date, exact run date unknown2026-11-02 · document_publicationCollaborative Intelligence Benchmark · original v1 evaluation
Claude Sonnet 5.567.9percentHigh reasoning effort; Table 5 overall across eight equally weighted dimensions; only tasks this model solves alone; 240 public+held-out tasks; three test seeds; fixed live partners and scripted teammate; LLM message routing with deterministic outcome checks; paper publication date, exact run date unknown2026-11-02 · document_publicationCollaborative Intelligence Benchmark · original v1 evaluation
GPT-6 Astra79.4percentHigh reasoning effort; Table 5 overall across eight equally weighted dimensions; only tasks this model solves alone; 240 public+held-out tasks; three test seeds; fixed live partners and scripted teammate; LLM message routing with deterministic outcome checks; paper publication date, exact run date unknown2026-11-02 · document_publicationCollaborative Intelligence Benchmark · original v1 evaluation
GPT-6.1 Sol78.7percentHigh reasoning effort; Table 5 overall across eight equally weighted dimensions; only tasks this model solves alone; 240 public+held-out tasks; three test seeds; fixed live partners and scripted teammate; LLM message routing with deterministic outcome checks; paper publication date, exact run date unknown2026-11-02 · document_publicationCollaborative Intelligence Benchmark · original v1 evaluation
Gemini 4 Argon82.2percentHigh reasoning effort; Table 5 overall across eight equally weighted dimensions; only tasks this model solves alone; 240 public+held-out tasks; three test seeds; fixed live partners and scripted teammate; LLM message routing with deterministic outcome checks; paper publication date, exact run date unknown2026-11-02 · document_publicationCollaborative Intelligence Benchmark · original v1 evaluation
Kimi K3.1 Pro56.1percentHigh reasoning effort; Table 5 overall across eight equally weighted dimensions; only tasks this model solves alone; 240 public+held-out tasks; three test seeds; fixed live partners and scripted teammate; LLM message routing with deterministic outcome checks; paper publication date, exact run date unknown2026-11-02 · document_publicationCollaborative Intelligence Benchmark · original v1 evaluation
Claude Fable 5.165.3percentLightest reasoning setting offered by the model API; Table 5 overall across eight equally weighted dimensions; only tasks this model solves alone; 240 public+held-out tasks; three test seeds; fixed live partners and scripted teammate; LLM message routing with deterministic outcome checks; paper publication date, exact run date unknown2026-11-02 · document_publicationCollaborative Intelligence Benchmark · original v1 evaluation
Claude Haiku 5.551.7percentLightest reasoning setting offered by the model API; Table 5 overall across eight equally weighted dimensions; only tasks this model solves alone; 240 public+held-out tasks; three test seeds; fixed live partners and scripted teammate; LLM message routing with deterministic outcome checks; paper publication date, exact run date unknown2026-11-02 · document_publicationCollaborative Intelligence Benchmark · original v1 evaluation
Claude Opus 5.571.0percentLightest reasoning setting offered by the model API; Table 5 overall across eight equally weighted dimensions; only tasks this model solves alone; 240 public+held-out tasks; three test seeds; fixed live partners and scripted teammate; LLM message routing with deterministic outcome checks; paper publication date, exact run date unknown2026-11-02 · document_publicationCollaborative Intelligence Benchmark · original v1 evaluation
Claude Sonnet 5.565.2percentLightest reasoning setting offered by the model API; Table 5 overall across eight equally weighted dimensions; only tasks this model solves alone; 240 public+held-out tasks; three test seeds; fixed live partners and scripted teammate; LLM message routing with deterministic outcome checks; paper publication date, exact run date unknown2026-11-02 · document_publicationCollaborative Intelligence Benchmark · original v1 evaluation
DeepSeek V4 Pro39.4percentLightest reasoning setting offered by the model API; Table 5 overall across eight equally weighted dimensions; only tasks this model solves alone; 240 public+held-out tasks; three test seeds; fixed live partners and scripted teammate; LLM message routing with deterministic outcome checks; paper publication date, exact run date unknown2026-11-02 · document_publicationCollaborative Intelligence Benchmark · original v1 evaluation
DeepSeek V4.1 Flash50.8percentLightest reasoning setting offered by the model API; Table 5 overall across eight equally weighted dimensions; only tasks this model solves alone; 240 public+held-out tasks; three test seeds; fixed live partners and scripted teammate; LLM message routing with deterministic outcome checks; paper publication date, exact run date unknown2026-11-02 · document_publicationCollaborative Intelligence Benchmark · original v1 evaluation
GLM-5.345.8percentLightest reasoning setting offered by the model API; Table 5 overall across eight equally weighted dimensions; only tasks this model solves alone; 240 public+held-out tasks; three test seeds; fixed live partners and scripted teammate; LLM message routing with deterministic outcome checks; paper publication date, exact run date unknown2026-11-02 · document_publicationCollaborative Intelligence Benchmark · original v1 evaluation
GPT-6 Astra76.5percentLightest reasoning setting offered by the model API; Table 5 overall across eight equally weighted dimensions; only tasks this model solves alone; 240 public+held-out tasks; three test seeds; fixed live partners and scripted teammate; LLM message routing with deterministic outcome checks; paper publication date, exact run date unknown2026-11-02 · document_publicationCollaborative Intelligence Benchmark · original v1 evaluation
GPT-6 Luna43.4percentLightest reasoning setting offered by the model API; Table 5 overall across eight equally weighted dimensions; only tasks this model solves alone; 240 public+held-out tasks; three test seeds; fixed live partners and scripted teammate; LLM message routing with deterministic outcome checks; paper publication date, exact run date unknown2026-11-02 · document_publicationCollaborative Intelligence Benchmark · original v1 evaluation
GPT-6.1 Sol74.5percentLightest reasoning setting offered by the model API; Table 5 overall across eight equally weighted dimensions; only tasks this model solves alone; 240 public+held-out tasks; three test seeds; fixed live partners and scripted teammate; LLM message routing with deterministic outcome checks; paper publication date, exact run date unknown2026-11-02 · document_publicationCollaborative Intelligence Benchmark · original v1 evaluation
Gemini 3.8 Flash51.5percentLightest reasoning setting offered by the model API; Table 5 overall across eight equally weighted dimensions; only tasks this model solves alone; 240 public+held-out tasks; three test seeds; fixed live partners and scripted teammate; LLM message routing with deterministic outcome checks; paper publication date, exact run date unknown2026-11-02 · document_publicationCollaborative Intelligence Benchmark · original v1 evaluation
Gemini 4 Argon76.6percentLightest reasoning setting offered by the model API; Table 5 overall across eight equally weighted dimensions; only tasks this model solves alone; 240 public+held-out tasks; three test seeds; fixed live partners and scripted teammate; LLM message routing with deterministic outcome checks; paper publication date, exact run date unknown2026-11-02 · document_publicationCollaborative Intelligence Benchmark · original v1 evaluation
Grok 4.755.6percentLightest reasoning setting offered by the model API; Table 5 overall across eight equally weighted dimensions; only tasks this model solves alone; 240 public+held-out tasks; three test seeds; fixed live partners and scripted teammate; LLM message routing with deterministic outcome checks; paper publication date, exact run date unknown2026-11-02 · document_publicationCollaborative Intelligence Benchmark · original v1 evaluation
Kimi K345.9percentLightest reasoning setting offered by the model API; Table 5 overall across eight equally weighted dimensions; only tasks this model solves alone; 240 public+held-out tasks; three test seeds; fixed live partners and scripted teammate; LLM message routing with deterministic outcome checks; paper publication date, exact run date unknown2026-11-02 · document_publicationCollaborative Intelligence Benchmark · original v1 evaluation
Kimi K3.1 Pro59.9percentLightest reasoning setting offered by the model API; Table 5 overall across eight equally weighted dimensions; only tasks this model solves alone; 240 public+held-out tasks; three test seeds; fixed live partners and scripted teammate; LLM message routing with deterministic outcome checks; paper publication date, exact run date unknown2026-11-02 · document_publicationCollaborative Intelligence Benchmark · original v1 evaluation
Mistral Large 444.5percentLightest reasoning setting offered by the model API; Table 5 overall across eight equally weighted dimensions; only tasks this model solves alone; 240 public+held-out tasks; three test seeds; fixed live partners and scripted teammate; LLM message routing with deterministic outcome checks; paper publication date, exact run date unknown2026-11-02 · document_publicationCollaborative Intelligence Benchmark · original v1 evaluation
Nemotron 3 Ultra30.6percentLightest reasoning setting offered by the model API; Table 5 overall across eight equally weighted dimensions; only tasks this model solves alone; 240 public+held-out tasks; three test seeds; fixed live partners and scripted teammate; LLM message routing with deterministic outcome checks; paper publication date, exact run date unknown2026-11-02 · document_publicationCollaborative Intelligence Benchmark · original v1 evaluation
gpt-oss-120b19.1percentLightest reasoning setting offered by the model API; Table 5 overall across eight equally weighted dimensions; only tasks this model solves alone; 240 public+held-out tasks; three test seeds; fixed live partners and scripted teammate; LLM message routing with deterministic outcome checks; paper publication date, exact run date unknown2026-11-02 · document_publicationCollaborative Intelligence Benchmark · original v1 evaluation

Explore related benchmarks

Suggestions based on shared evidence or source categories, not verified equivalents.

PersonaCodeBenchShared source categories →xDailyBenchShared source categories →Blueprint-Bench 2Shared source categories →DSBench-FullStackShared source categories →Cohere Memory Usage QualityShared source categories →QwenQoderBenchShared source categories →RecreationBenchShared source categories →DeepPlanningShared source categories →

Put this benchmark in context

Tools and agents benchmarksSafety and reliability benchmarks

Continue exploring

Search the catalog using this record’s source labels. Search matches are not verified equivalents.

agents →Model reports · all records →