Model reports

SlackBench

Can an agent answer a workplace question by checking conversations against code and documents? Inspect the tool interface as well as the model.

Original paper Official dataset Catalog source

AI Benchmark Radar / Equality Machine · Reviewed 2026-11-05

Paper v1 published 3 November 2026; reviewed 5 November. All 33 main configurations plus seven CLI-access and ten reasoning variants; original category scores and diagnostics remain separate.

Cross-check a shared workspace

140 questions span search (58), abstention (48), summarization (14) and circumstance inference (20). Reenactments contain 15 channels, about 3,000 messages, 14 repositories and nine documents.

SlackBench · original paper v1

Four categories have equal weight

GPT-5.4 judges binary answers; summaries receive checklist credit. Three runs are averaged. Overall weights categories equally despite unequal task counts.

SlackBench · original paper v1

Preserve the tool interface

Main runs use medium reasoning and a 20-minute query cap. CLI-access and reasoning variants are separate. Actual run dates and endpoint revisions are unspecified.

SlackBench · original paper v1

Read the evidence boundaries

These are reconstructed academic workflows. Some reference answers are withheld. Human baselines allow AI assistance; judge agreement covers the sampled outputs, not every future task.

SlackBench · original paper v1

All main configurations

Original reported values.

All main configurations
Harness	Model	AS %	HA %	SU %	CI %	Overall %
Antigravity	Gemini 4 Argon	70.69	90.28	76.68	83.33	80.25
Muse Code	Muse Wonder	81.03	62.50	71.42	80.00	73.74
Claude Code	Claude Fable 5.1	67.82	71.53	71.98	83.33	73.67
Claude Code	Claude Opus 5.5	71.84	75.00	70.42	73.33	72.65
Claude Code	Muse Wonder	68.97	66.67	67.25	80.00	70.72
Claude Code	GLM 5.3	69.54	70.83	65.12	58.33	65.96
Grok Build	Grok 4.7	64.94	56.94	70.01	70.00	65.47
Muse Code	Muse Spark 1.3	66.67	52.08	67.40	68.33	63.62
Codex CLI	GPT 6.1 Terra	63.22	55.56	64.05	61.67	61.12
Codex CLI	GPT 6 Astra	62.07	54.17	65.67	60.00	60.48
Codex CLI	GPT 6.1 Sol	59.77	50.69	62.30	60.00	58.19
Claude Code	Claude Opus 4.7	52.87	60.42	52.69	60.00	56.49
OpenHands	GPT-5.5	60.92	25.69	59.30	61.67	51.89
Terminus-2	GPT-5.5	56.32	21.53	48.72	66.67	48.31
Gemini CLI	Gemini 3.1 Pro Preview	54.02	42.36	38.74	50.00	46.28
Claude Code	Claude Sonnet 4.6	50.00	40.97	47.66	45.00	45.91
OpenHands	Claude Opus 4.6	48.28	15.28	53.24	66.67	45.87
Terminus-2	Claude Opus 4.6	54.60	11.81	56.71	60.00	45.78
Codex CLI	GPT-5.5	55.17	18.75	53.14	55.00	45.52
Claude Code	Kimi K2.6	50.00	32.64	49.11	46.67	44.60
Claude Code	Claude Opus 4.6	52.87	20.83	51.39	41.67	41.69
Terminus-2	Gemini 3.1 Pro Preview	50.00	34.72	27.35	50.00	40.52
Claude Code	DeepSeek V4 Pro	43.68	37.50	47.93	30.00	39.78
OpenHands	Gemini 3.1 Pro Preview	44.25	31.25	36.36	43.33	38.80
Terminus-2	Claude Sonnet 4.6	51.15	15.28	44.95	43.33	38.68
Claude Code	Qwen3.7 Max	36.78	38.19	41.23	36.67	38.22
Claude Code	GLM-5.1	50.00	22.22	50.09	30.00	38.08
Codex CLI	GPT-5.4	51.15	21.53	40.42	36.67	37.44
Codex CLI	GPT 6 Luna	45.98	22.92	44.05	36.67	37.40
Gemini CLI	Gemini 3 Flash Preview	42.53	1.39	27.26	53.33	31.13
Claude Code	Claude Haiku 4.5	38.51	26.39	30.15	15.00	27.51
Codex CLI	GPT-5.4-Mini	36.21	13.89	34.73	23.33	27.04
Terminus-2	GPT-5.4-Mini	22.41	5.56	16.63	6.67	12.82

Medium reasoning; SlackBench mock MCP interface.

SlackBench · main-results

Workspace access comparison

Original reported values.

Workspace access comparison
Harness	Model	Access	AS %	HA %	SU %	CI %	Overall %
Claude Code	Claude Opus 5.5	CLI	66.67	72.92	70.12	85.00	73.68
Claude Code	Claude Opus 5.5	MCP	71.84	75.00	70.42	73.33	72.65
Claude Code	Claude Opus 4.7	CLI	57.47	55.56	60.48	65.00	59.63
Claude Code	Claude Opus 4.7	MCP	52.87	60.42	52.69	60.00	56.49
Claude Code	Claude Sonnet 4.6	CLI	58.62	41.67	55.52	63.33	54.78
Claude Code	Claude Sonnet 4.6	MCP	50.00	40.97	47.66	45.00	45.91
Codex CLI	GPT-5.5	CLI	72.42	27.78	67.73	71.67	59.90
Codex CLI	GPT-5.5	MCP	55.17	18.75	53.14	55.00	45.52
Codex CLI	GPT-5.4-Mini	CLI	53.45	18.75	48.62	51.67	43.12
Codex CLI	GPT-5.4-Mini	MCP	36.21	13.89	34.73	23.33	27.04
Gemini CLI	Gemini 3.1 Pro Preview	CLI	62.07	37.50	51.01	66.67	54.31
Gemini CLI	Gemini 3.1 Pro Preview	MCP	54.02	42.36	38.74	50.00	46.28
Gemini CLI	Gemini 3 Flash Preview	CLI	42.53	3.47	33.86	58.33	34.55
Gemini CLI	Gemini 3 Flash Preview	MCP	42.53	1.39	27.26	53.33	31.13

MCP rows repeat the corresponding main baseline; they are not counted as new observations.

SlackBench · access-results

Reasoning effort comparison

Original reported values.

Reasoning effort comparison
Harness	Model	Effort	AS %	HA %	SU %	CI %	Overall %
Claude Code	Claude Opus 4.7	Low	50.00	50.00	45.19	50.00	48.80
Claude Code	Claude Opus 4.7	Medium	52.87	60.42	52.69	60.00	56.49
Claude Code	Claude Opus 4.7	High	58.62	72.92	63.94	75.00	67.62
Claude Code	Claude Sonnet 4.6	Low	39.66	41.67	40.86	55.00	44.30
Claude Code	Claude Sonnet 4.6	Medium	50.00	40.97	47.66	45.00	45.91
Claude Code	Claude Sonnet 4.6	High	55.17	33.33	44.96	60.00	48.37
Codex CLI	GPT-5.5	Low	48.28	18.75	51.52	45.00	40.89
Codex CLI	GPT-5.5	Medium	55.17	18.75	53.14	55.00	45.52
Codex CLI	GPT-5.5	High	58.62	25.00	60.90	65.00	52.38
Codex CLI	GPT-5.4-mini	Low	25.86	10.41	24.80	0.00	15.27
Codex CLI	GPT-5.4-mini	Medium	36.21	13.89	34.73	23.33	27.04
Codex CLI	GPT-5.4-mini	High	41.38	10.42	38.09	20.00	27.47
Gemini CLI	Gemini 3.1 Pro Preview	Low	43.10	37.50	27.04	45.00	38.16
Gemini CLI	Gemini 3.1 Pro Preview	Medium	54.02	42.36	38.74	50.00	46.28
Gemini CLI	Gemini 3.1 Pro Preview	High	58.62	37.50	39.83	55.00	47.74

Medium rows repeat the main baseline.

SlackBench · effort-results

Filtered official-tool comparison

Original reported values.

Filtered official-tool comparison
Harness	Model	Access	AS filtered %	HA filtered %
Claude Code	Claude Opus 4.7	Ours	53.33	53.19
Claude Code	Claude Opus 4.7	Official MCP	57.78	59.57
Claude Code	Claude Sonnet 4.6	Ours	46.67	34.04
Claude Code	Claude Sonnet 4.6	Official MCP	57.78	38.30
Codex CLI	GPT-5.5	Ours	53.33	18.09
Codex CLI	GPT-5.5	Official MCP	55.56	25.53
Codex CLI	GPT-5.4-Mini	Ours	46.67	12.77
Codex CLI	GPT-5.4-Mini	Official MCP	46.67	19.15

Only filtered AS/HA queries; these are not full-benchmark overall scores.

SlackBench · official-tools

Pairwise judge agreement

Original reported values.

Pairwise judge agreement
Judge	GPT-5.4 %	Opus 4.7 %	Gemini 3.1 Pro %
GPT-5.4	100%	95.1%	96.3%
Opus 4.7	–	100%	96.6%
Gemini 3.1 Pro	–	–	100%

AS, HA and CI answers from six specified agent configurations; not task accuracy.

SlackBench · judge-agreement

AI-assisted human baseline

Three independent researchers; reported mean and interval.

Human reference
Overall %	AS %	HA %	SU %	CI %	95% bootstrap CI
84.5	91	90	82	75	77.0–92.0

AI assistance was permitted; not an unaided-human comparison.

SlackBench · human

Original publication

Paper v1 published 3 November 2026; reviewed 5 November. All 33 main configurations plus seven CLI-access and ten reasoning variants; original category scores and diagnostics remain separate.

Reviewed 2026-11-05

Evidence available for this benchmark

This source record contains 50 numeric observations. The source distinguishes 24 model identities or configurations. A model may appear more than once; check the version, tools and run budget before comparing. Read results and row-level evidence →

Reported metric: overall accuracy (unweighted mean of four category scores)
Recorded unit: percent
Metric direction: higher is better

Start with these resources

Catalog source · arxiv.org

Cited documents and dates (1)

SlackBench · original v1 evaluation · 2026-11-03

Evaluation context and evidence

Source: Model reports
Publisher: Not recorded
Benchmark release: 2026-11-03
Collected: Not recorded
Modality: Not recorded

These are source-reported observations, not independent reruns by this site. Check the task version, evaluation protocol, unit and reporting date before comparing models. Results from different setups are not a universal ranking.

Reported results 50

Shown in source observation order, not ranked. Dates retain the precision provided by the source.

Model	Value	Unit	Protocol	Reported date and basis	Source
Gemini 4 Argon	80.25	percent	Antigravity harness; SlackBench mock MCP workspace access; medium reasoning; Harbor, 20-minute query limit; average of three runs and equal mean of AS/HA/SU/CI; GPT-5.4 judge, SU partial checklist credit; read-only Slack/GitHub tools plus terminal and web search; original publication date, exact run dates/API revisions unknown	2026-11-03 · document_publication	SlackBench · original v1 evaluation
Claude Opus 4.7	67.62	percent	Claude Code harness; SlackBench mock MCP workspace access; high reasoning; Harbor, 20-minute query limit; average of three runs and equal mean of AS/HA/SU/CI; GPT-5.4 judge, SU partial checklist credit; read-only Slack/GitHub tools plus terminal and web search; original publication date, exact run dates/API revisions unknown	2026-11-03 · document_publication	SlackBench · original v1 evaluation
Claude Sonnet 4.6	48.37	percent	Claude Code harness; SlackBench mock MCP workspace access; high reasoning; Harbor, 20-minute query limit; average of three runs and equal mean of AS/HA/SU/CI; GPT-5.4 judge, SU partial checklist credit; read-only Slack/GitHub tools plus terminal and web search; original publication date, exact run dates/API revisions unknown	2026-11-03 · document_publication	SlackBench · original v1 evaluation
Claude Opus 4.7	48.8	percent	Claude Code harness; SlackBench mock MCP workspace access; low reasoning; Harbor, 20-minute query limit; average of three runs and equal mean of AS/HA/SU/CI; GPT-5.4 judge, SU partial checklist credit; read-only Slack/GitHub tools plus terminal and web search; original publication date, exact run dates/API revisions unknown	2026-11-03 · document_publication	SlackBench · original v1 evaluation
Claude Sonnet 4.6	44.3	percent	Claude Code harness; SlackBench mock MCP workspace access; low reasoning; Harbor, 20-minute query limit; average of three runs and equal mean of AS/HA/SU/CI; GPT-5.4 judge, SU partial checklist credit; read-only Slack/GitHub tools plus terminal and web search; original publication date, exact run dates/API revisions unknown	2026-11-03 · document_publication	SlackBench · original v1 evaluation
Claude Fable 5.1	73.67	percent	Claude Code harness; SlackBench mock MCP workspace access; medium reasoning; Harbor, 20-minute query limit; average of three runs and equal mean of AS/HA/SU/CI; GPT-5.4 judge, SU partial checklist credit; read-only Slack/GitHub tools plus terminal and web search; original publication date, exact run dates/API revisions unknown	2026-11-03 · document_publication	SlackBench · original v1 evaluation
Claude Haiku 4.5	27.51	percent	Claude Code harness; SlackBench mock MCP workspace access; medium reasoning; Harbor, 20-minute query limit; average of three runs and equal mean of AS/HA/SU/CI; GPT-5.4 judge, SU partial checklist credit; read-only Slack/GitHub tools plus terminal and web search; original publication date, exact run dates/API revisions unknown	2026-11-03 · document_publication	SlackBench · original v1 evaluation
Claude Opus 4.6	41.69	percent	Claude Code harness; SlackBench mock MCP workspace access; medium reasoning; Harbor, 20-minute query limit; average of three runs and equal mean of AS/HA/SU/CI; GPT-5.4 judge, SU partial checklist credit; read-only Slack/GitHub tools plus terminal and web search; original publication date, exact run dates/API revisions unknown	2026-11-03 · document_publication	SlackBench · original v1 evaluation
Claude Opus 4.7	56.49	percent	Claude Code harness; SlackBench mock MCP workspace access; medium reasoning; Harbor, 20-minute query limit; average of three runs and equal mean of AS/HA/SU/CI; GPT-5.4 judge, SU partial checklist credit; read-only Slack/GitHub tools plus terminal and web search; original publication date, exact run dates/API revisions unknown	2026-11-03 · document_publication	SlackBench · original v1 evaluation
Claude Opus 5.5	72.65	percent	Claude Code harness; SlackBench mock MCP workspace access; medium reasoning; Harbor, 20-minute query limit; average of three runs and equal mean of AS/HA/SU/CI; GPT-5.4 judge, SU partial checklist credit; read-only Slack/GitHub tools plus terminal and web search; original publication date, exact run dates/API revisions unknown	2026-11-03 · document_publication	SlackBench · original v1 evaluation
Claude Sonnet 4.6	45.91	percent	Claude Code harness; SlackBench mock MCP workspace access; medium reasoning; Harbor, 20-minute query limit; average of three runs and equal mean of AS/HA/SU/CI; GPT-5.4 judge, SU partial checklist credit; read-only Slack/GitHub tools plus terminal and web search; original publication date, exact run dates/API revisions unknown	2026-11-03 · document_publication	SlackBench · original v1 evaluation
DeepSeek V4 Pro	39.78	percent	Claude Code harness; SlackBench mock MCP workspace access; medium reasoning; Harbor, 20-minute query limit; average of three runs and equal mean of AS/HA/SU/CI; GPT-5.4 judge, SU partial checklist credit; read-only Slack/GitHub tools plus terminal and web search; original publication date, exact run dates/API revisions unknown	2026-11-03 · document_publication	SlackBench · original v1 evaluation
GLM 5.3	65.96	percent	Claude Code harness; SlackBench mock MCP workspace access; medium reasoning; Harbor, 20-minute query limit; average of three runs and equal mean of AS/HA/SU/CI; GPT-5.4 judge, SU partial checklist credit; read-only Slack/GitHub tools plus terminal and web search; original publication date, exact run dates/API revisions unknown	2026-11-03 · document_publication	SlackBench · original v1 evaluation
GLM-5.1	38.08	percent	Claude Code harness; SlackBench mock MCP workspace access; medium reasoning; Harbor, 20-minute query limit; average of three runs and equal mean of AS/HA/SU/CI; GPT-5.4 judge, SU partial checklist credit; read-only Slack/GitHub tools plus terminal and web search; original publication date, exact run dates/API revisions unknown	2026-11-03 · document_publication	SlackBench · original v1 evaluation
Kimi K2.6	44.6	percent	Claude Code harness; SlackBench mock MCP workspace access; medium reasoning; Harbor, 20-minute query limit; average of three runs and equal mean of AS/HA/SU/CI; GPT-5.4 judge, SU partial checklist credit; read-only Slack/GitHub tools plus terminal and web search; original publication date, exact run dates/API revisions unknown	2026-11-03 · document_publication	SlackBench · original v1 evaluation
Muse Wonder	70.72	percent	Claude Code harness; SlackBench mock MCP workspace access; medium reasoning; Harbor, 20-minute query limit; average of three runs and equal mean of AS/HA/SU/CI; GPT-5.4 judge, SU partial checklist credit; read-only Slack/GitHub tools plus terminal and web search; original publication date, exact run dates/API revisions unknown	2026-11-03 · document_publication	SlackBench · original v1 evaluation
Qwen3.7 Max	38.22	percent	Claude Code harness; SlackBench mock MCP workspace access; medium reasoning; Harbor, 20-minute query limit; average of three runs and equal mean of AS/HA/SU/CI; GPT-5.4 judge, SU partial checklist credit; read-only Slack/GitHub tools plus terminal and web search; original publication date, exact run dates/API revisions unknown	2026-11-03 · document_publication	SlackBench · original v1 evaluation
Claude Opus 4.7	59.63	percent	Claude Code harness; direct terminal access to exported Slack workspace; medium reasoning; Harbor, 20-minute query limit; average of three runs and equal mean of AS/HA/SU/CI; GPT-5.4 judge, SU partial checklist credit; read-only Slack/GitHub tools plus terminal and web search; original publication date, exact run dates/API revisions unknown	2026-11-03 · document_publication	SlackBench · original v1 evaluation
Claude Opus 5.5	73.68	percent	Claude Code harness; direct terminal access to exported Slack workspace; medium reasoning; Harbor, 20-minute query limit; average of three runs and equal mean of AS/HA/SU/CI; GPT-5.4 judge, SU partial checklist credit; read-only Slack/GitHub tools plus terminal and web search; original publication date, exact run dates/API revisions unknown	2026-11-03 · document_publication	SlackBench · original v1 evaluation
Claude Sonnet 4.6	54.78	percent	Claude Code harness; direct terminal access to exported Slack workspace; medium reasoning; Harbor, 20-minute query limit; average of three runs and equal mean of AS/HA/SU/CI; GPT-5.4 judge, SU partial checklist credit; read-only Slack/GitHub tools plus terminal and web search; original publication date, exact run dates/API revisions unknown	2026-11-03 · document_publication	SlackBench · original v1 evaluation
GPT-5.4-Mini	27.47	percent	Codex CLI harness; SlackBench mock MCP workspace access; high reasoning; Harbor, 20-minute query limit; average of three runs and equal mean of AS/HA/SU/CI; GPT-5.4 judge, SU partial checklist credit; read-only Slack/GitHub tools plus terminal and web search; original publication date, exact run dates/API revisions unknown	2026-11-03 · document_publication	SlackBench · original v1 evaluation
GPT-5.5	52.38	percent	Codex CLI harness; SlackBench mock MCP workspace access; high reasoning; Harbor, 20-minute query limit; average of three runs and equal mean of AS/HA/SU/CI; GPT-5.4 judge, SU partial checklist credit; read-only Slack/GitHub tools plus terminal and web search; original publication date, exact run dates/API revisions unknown	2026-11-03 · document_publication	SlackBench · original v1 evaluation
GPT-5.4-Mini	15.27	percent	Codex CLI harness; SlackBench mock MCP workspace access; low reasoning; Harbor, 20-minute query limit; average of three runs and equal mean of AS/HA/SU/CI; GPT-5.4 judge, SU partial checklist credit; read-only Slack/GitHub tools plus terminal and web search; original publication date, exact run dates/API revisions unknown	2026-11-03 · document_publication	SlackBench · original v1 evaluation
GPT-5.5	40.89	percent	Codex CLI harness; SlackBench mock MCP workspace access; low reasoning; Harbor, 20-minute query limit; average of three runs and equal mean of AS/HA/SU/CI; GPT-5.4 judge, SU partial checklist credit; read-only Slack/GitHub tools plus terminal and web search; original publication date, exact run dates/API revisions unknown	2026-11-03 · document_publication	SlackBench · original v1 evaluation
GPT 6 Astra	60.48	percent	Codex CLI harness; SlackBench mock MCP workspace access; medium reasoning; Harbor, 20-minute query limit; average of three runs and equal mean of AS/HA/SU/CI; GPT-5.4 judge, SU partial checklist credit; read-only Slack/GitHub tools plus terminal and web search; original publication date, exact run dates/API revisions unknown	2026-11-03 · document_publication	SlackBench · original v1 evaluation
GPT 6 Luna	37.4	percent	Codex CLI harness; SlackBench mock MCP workspace access; medium reasoning; Harbor, 20-minute query limit; average of three runs and equal mean of AS/HA/SU/CI; GPT-5.4 judge, SU partial checklist credit; read-only Slack/GitHub tools plus terminal and web search; original publication date, exact run dates/API revisions unknown	2026-11-03 · document_publication	SlackBench · original v1 evaluation
GPT 6.1 Sol	58.19	percent	Codex CLI harness; SlackBench mock MCP workspace access; medium reasoning; Harbor, 20-minute query limit; average of three runs and equal mean of AS/HA/SU/CI; GPT-5.4 judge, SU partial checklist credit; read-only Slack/GitHub tools plus terminal and web search; original publication date, exact run dates/API revisions unknown	2026-11-03 · document_publication	SlackBench · original v1 evaluation
GPT 6.1 Terra	61.12	percent	Codex CLI harness; SlackBench mock MCP workspace access; medium reasoning; Harbor, 20-minute query limit; average of three runs and equal mean of AS/HA/SU/CI; GPT-5.4 judge, SU partial checklist credit; read-only Slack/GitHub tools plus terminal and web search; original publication date, exact run dates/API revisions unknown	2026-11-03 · document_publication	SlackBench · original v1 evaluation
GPT-5.4	37.44	percent	Codex CLI harness; SlackBench mock MCP workspace access; medium reasoning; Harbor, 20-minute query limit; average of three runs and equal mean of AS/HA/SU/CI; GPT-5.4 judge, SU partial checklist credit; read-only Slack/GitHub tools plus terminal and web search; original publication date, exact run dates/API revisions unknown	2026-11-03 · document_publication	SlackBench · original v1 evaluation
GPT-5.4-Mini	27.04	percent	Codex CLI harness; SlackBench mock MCP workspace access; medium reasoning; Harbor, 20-minute query limit; average of three runs and equal mean of AS/HA/SU/CI; GPT-5.4 judge, SU partial checklist credit; read-only Slack/GitHub tools plus terminal and web search; original publication date, exact run dates/API revisions unknown	2026-11-03 · document_publication	SlackBench · original v1 evaluation
GPT-5.5	45.52	percent	Codex CLI harness; SlackBench mock MCP workspace access; medium reasoning; Harbor, 20-minute query limit; average of three runs and equal mean of AS/HA/SU/CI; GPT-5.4 judge, SU partial checklist credit; read-only Slack/GitHub tools plus terminal and web search; original publication date, exact run dates/API revisions unknown	2026-11-03 · document_publication	SlackBench · original v1 evaluation
GPT-5.4-Mini	43.12	percent	Codex CLI harness; direct terminal access to exported Slack workspace; medium reasoning; Harbor, 20-minute query limit; average of three runs and equal mean of AS/HA/SU/CI; GPT-5.4 judge, SU partial checklist credit; read-only Slack/GitHub tools plus terminal and web search; original publication date, exact run dates/API revisions unknown	2026-11-03 · document_publication	SlackBench · original v1 evaluation
GPT-5.5	59.9	percent	Codex CLI harness; direct terminal access to exported Slack workspace; medium reasoning; Harbor, 20-minute query limit; average of three runs and equal mean of AS/HA/SU/CI; GPT-5.4 judge, SU partial checklist credit; read-only Slack/GitHub tools plus terminal and web search; original publication date, exact run dates/API revisions unknown	2026-11-03 · document_publication	SlackBench · original v1 evaluation
Gemini 3.1 Pro Preview	47.74	percent	Gemini CLI harness; SlackBench mock MCP workspace access; high reasoning; Harbor, 20-minute query limit; average of three runs and equal mean of AS/HA/SU/CI; GPT-5.4 judge, SU partial checklist credit; read-only Slack/GitHub tools plus terminal and web search; original publication date, exact run dates/API revisions unknown	2026-11-03 · document_publication	SlackBench · original v1 evaluation
Gemini 3.1 Pro Preview	38.16	percent	Gemini CLI harness; SlackBench mock MCP workspace access; low reasoning; Harbor, 20-minute query limit; average of three runs and equal mean of AS/HA/SU/CI; GPT-5.4 judge, SU partial checklist credit; read-only Slack/GitHub tools plus terminal and web search; original publication date, exact run dates/API revisions unknown	2026-11-03 · document_publication	SlackBench · original v1 evaluation
Gemini 3 Flash Preview	31.13	percent	Gemini CLI harness; SlackBench mock MCP workspace access; medium reasoning; Harbor, 20-minute query limit; average of three runs and equal mean of AS/HA/SU/CI; GPT-5.4 judge, SU partial checklist credit; read-only Slack/GitHub tools plus terminal and web search; original publication date, exact run dates/API revisions unknown	2026-11-03 · document_publication	SlackBench · original v1 evaluation
Gemini 3.1 Pro Preview	46.28	percent	Gemini CLI harness; SlackBench mock MCP workspace access; medium reasoning; Harbor, 20-minute query limit; average of three runs and equal mean of AS/HA/SU/CI; GPT-5.4 judge, SU partial checklist credit; read-only Slack/GitHub tools plus terminal and web search; original publication date, exact run dates/API revisions unknown	2026-11-03 · document_publication	SlackBench · original v1 evaluation
Gemini 3 Flash Preview	34.55	percent	Gemini CLI harness; direct terminal access to exported Slack workspace; medium reasoning; Harbor, 20-minute query limit; average of three runs and equal mean of AS/HA/SU/CI; GPT-5.4 judge, SU partial checklist credit; read-only Slack/GitHub tools plus terminal and web search; original publication date, exact run dates/API revisions unknown	2026-11-03 · document_publication	SlackBench · original v1 evaluation
Gemini 3.1 Pro Preview	54.31	percent	Gemini CLI harness; direct terminal access to exported Slack workspace; medium reasoning; Harbor, 20-minute query limit; average of three runs and equal mean of AS/HA/SU/CI; GPT-5.4 judge, SU partial checklist credit; read-only Slack/GitHub tools plus terminal and web search; original publication date, exact run dates/API revisions unknown	2026-11-03 · document_publication	SlackBench · original v1 evaluation
Grok 4.7	65.47	percent	Grok Build harness; SlackBench mock MCP workspace access; medium reasoning; Harbor, 20-minute query limit; average of three runs and equal mean of AS/HA/SU/CI; GPT-5.4 judge, SU partial checklist credit; read-only Slack/GitHub tools plus terminal and web search; original publication date, exact run dates/API revisions unknown	2026-11-03 · document_publication	SlackBench · original v1 evaluation
Muse Spark 1.3	63.62	percent	Muse Code harness; SlackBench mock MCP workspace access; medium reasoning; Harbor, 20-minute query limit; average of three runs and equal mean of AS/HA/SU/CI; GPT-5.4 judge, SU partial checklist credit; read-only Slack/GitHub tools plus terminal and web search; original publication date, exact run dates/API revisions unknown	2026-11-03 · document_publication	SlackBench · original v1 evaluation
Muse Wonder	73.74	percent	Muse Code harness; SlackBench mock MCP workspace access; medium reasoning; Harbor, 20-minute query limit; average of three runs and equal mean of AS/HA/SU/CI; GPT-5.4 judge, SU partial checklist credit; read-only Slack/GitHub tools plus terminal and web search; original publication date, exact run dates/API revisions unknown	2026-11-03 · document_publication	SlackBench · original v1 evaluation
Claude Opus 4.6	45.87	percent	OpenHands harness; SlackBench mock MCP workspace access; medium reasoning; Harbor, 20-minute query limit; average of three runs and equal mean of AS/HA/SU/CI; GPT-5.4 judge, SU partial checklist credit; read-only Slack/GitHub tools plus terminal and web search; original publication date, exact run dates/API revisions unknown	2026-11-03 · document_publication	SlackBench · original v1 evaluation
GPT-5.5	51.89	percent	OpenHands harness; SlackBench mock MCP workspace access; medium reasoning; Harbor, 20-minute query limit; average of three runs and equal mean of AS/HA/SU/CI; GPT-5.4 judge, SU partial checklist credit; read-only Slack/GitHub tools plus terminal and web search; original publication date, exact run dates/API revisions unknown	2026-11-03 · document_publication	SlackBench · original v1 evaluation
Gemini 3.1 Pro Preview	38.8	percent	OpenHands harness; SlackBench mock MCP workspace access; medium reasoning; Harbor, 20-minute query limit; average of three runs and equal mean of AS/HA/SU/CI; GPT-5.4 judge, SU partial checklist credit; read-only Slack/GitHub tools plus terminal and web search; original publication date, exact run dates/API revisions unknown	2026-11-03 · document_publication	SlackBench · original v1 evaluation
Claude Opus 4.6	45.78	percent	Terminus-2 harness; SlackBench mock MCP workspace access; medium reasoning; Harbor, 20-minute query limit; average of three runs and equal mean of AS/HA/SU/CI; GPT-5.4 judge, SU partial checklist credit; read-only Slack/GitHub tools plus terminal and web search; original publication date, exact run dates/API revisions unknown	2026-11-03 · document_publication	SlackBench · original v1 evaluation
Claude Sonnet 4.6	38.68	percent	Terminus-2 harness; SlackBench mock MCP workspace access; medium reasoning; Harbor, 20-minute query limit; average of three runs and equal mean of AS/HA/SU/CI; GPT-5.4 judge, SU partial checklist credit; read-only Slack/GitHub tools plus terminal and web search; original publication date, exact run dates/API revisions unknown	2026-11-03 · document_publication	SlackBench · original v1 evaluation
GPT-5.4-Mini	12.82	percent	Terminus-2 harness; SlackBench mock MCP workspace access; medium reasoning; Harbor, 20-minute query limit; average of three runs and equal mean of AS/HA/SU/CI; GPT-5.4 judge, SU partial checklist credit; read-only Slack/GitHub tools plus terminal and web search; original publication date, exact run dates/API revisions unknown	2026-11-03 · document_publication	SlackBench · original v1 evaluation
GPT-5.5	48.31	percent	Terminus-2 harness; SlackBench mock MCP workspace access; medium reasoning; Harbor, 20-minute query limit; average of three runs and equal mean of AS/HA/SU/CI; GPT-5.4 judge, SU partial checklist credit; read-only Slack/GitHub tools plus terminal and web search; original publication date, exact run dates/API revisions unknown	2026-11-03 · document_publication	SlackBench · original v1 evaluation
Gemini 3.1 Pro Preview	40.52	percent	Terminus-2 harness; SlackBench mock MCP workspace access; medium reasoning; Harbor, 20-minute query limit; average of three runs and equal mean of AS/HA/SU/CI; GPT-5.4 judge, SU partial checklist credit; read-only Slack/GitHub tools plus terminal and web search; original publication date, exact run dates/API revisions unknown	2026-11-03 · document_publication	SlackBench · original v1 evaluation

Cross-check a shared workspace

Four categories have equal weight

Preserve the tool interface

Read the evidence boundaries

All main configurations

Workspace access comparison

Reasoning effort comparison

Filtered official-tool comparison

Pairwise judge agreement

AI-assisted human baseline

Original publication

Evidence available for this benchmark

Start with these resources

Evaluation context and evidence

Reported results 50

Explore related benchmarks

Put this benchmark in context

Continue exploring