AI EVALUATION, IN CONTEXT.BY EQUALITY MACHINE
Benchmark Radar.Benchmarks. Model results. Original sources.
/

Artificial Analysis

AutomationBench-AA

Agentic SaaS workflows

Evaluation context and evidence

Source
Artificial Analysis
Publisher
Not recorded
Benchmark release
Not recorded
Collected
2026-08-25
Modality
Not recorded

These are source-reported observations, not independent reruns by this site. Check the task version, evaluation protocol, unit and reporting date before comparing models. Results from different setups are not a universal ranking.

Original sources

Reported results 39

Shown in source observation order, not ranked. Dates retain the precision provided by the source.

ModelValueUnitProtocolReported date and basisSource
Gemini 3.7 Flash (high)0.6274556199710792Not recordedNot recorded2026-08-13 · model_announcementArtificial Analysis
Kimi K3 (max)0.5270588443066796Not recordedNot recorded2026-07-16 · model_announcementArtificial Analysis
Grok 4.5 (high)0.5143740721186943Not recordedNot recorded2026-07-08 · model_announcementArtificial Analysis
GPT-5.6 Sol (max)0.5118769513495001Not recordedNot recorded2026-07-09 · model_announcementArtificial Analysis
Gemini 3.6 Flash (high)0.5107699915638565Not recordedNot recorded2026-07-21 · model_announcementArtificial Analysis
Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback)0.48581851262754977Not recordedNot recorded2026-06-09 · model_announcementArtificial Analysis
Claude Opus 4.8 (Adaptive Reasoning, Max Effort)0.48529386173175704Not recordedNot recorded2026-05-28 · model_announcementArtificial Analysis
GPT-5.6 Terra (max)0.4561448924929106Not recordedNot recorded2026-07-09 · model_announcementArtificial Analysis
Gemini 3.5 Flash (high)0.42637546323234665Not recordedNot recorded2026-05-19 · model_announcementArtificial Analysis
GPT-5.6 Luna (max)0.4224225837450071Not recordedNot recorded2026-07-09 · model_announcementArtificial Analysis
GPT-5.5 (xhigh)0.42103132391668596Not recordedNot recorded2026-04-23 · model_announcementArtificial Analysis
GPT-5.5 (high)0.4016408791094086Not recordedNot recorded2026-04-23 · model_announcementArtificial Analysis
Claude Sonnet 5 (Adaptive Reasoning, Max Effort)0.39190097119704015Not recordedNot recorded2026-06-30 · model_announcementArtificial Analysis
GPT-5.5 (medium)0.37809962422121907Not recordedNot recorded2026-04-23 · model_announcementArtificial Analysis
Gemini 3.1 Pro Preview0.37535846144401597Not recordedNot recorded2026-02-19 · model_announcementArtificial Analysis
Gemini 3.5 Flash-Lite0.3265910091801892Not recordedNot recorded2026-07-21 · model_announcementArtificial Analysis
GLM-5.2 (max)0.27844625079401897Not recordedNot recorded2026-06-16 · model_announcementArtificial Analysis
Qwen3.7 Max0.2557589370521027Not recordedNot recorded2026-05-19 · model_announcementArtificial Analysis
Claude Sonnet 4.6 (Adaptive Reasoning, Max Effort)0.23823599456385894Not recordedNot recorded2026-02-17 · model_announcementArtificial Analysis
Kimi K2.7 Code0.22528586634678724Not recordedNot recorded2026-06-12 · model_announcementArtificial Analysis
Qwen3.7 Plus0.20433540292579966Not recordedNot recorded2026-06-01 · model_announcementArtificial Analysis
Kimi K2.60.19552711052930297Not recordedNot recorded2026-04-20 · model_announcementArtificial Analysis
Nex-N2-Pro0.194033175065174Not recordedNot recorded2026-06-02 · model_announcementArtificial Analysis
DeepSeek V4 Pro (Reasoning, Max Effort)0.18827715939133852Not recordedNot recorded2026-04-24 · model_announcementArtificial Analysis
Qwen3.6 Plus0.18645616010518853Not recordedNot recorded2026-04-02 · model_announcementArtificial Analysis
MiMo-V2.5-Pro0.1686244634925501Not recordedNot recorded2026-04-22 · model_announcementArtificial Analysis
DeepSeek V4 Flash (Reasoning, Max Effort)0.16582172712070506Not recordedNot recorded2026-04-24 · model_announcementArtificial Analysis
MiniMax-M30.1589777434670109Not recordedNot recorded2026-06-01 · model_announcementArtificial Analysis
Mistral Medium 3.50.13655123642959935Not recordedNot recorded2026-04-29 · model_announcementArtificial Analysis
Qwen3.5 397B A17B (Reasoning)0.12195605269463661Not recordedNot recorded2026-02-16 · model_announcementArtificial Analysis
Gemini 3.1 Flash-Lite0.10641533320288254Not recordedNot recorded2026-03-03 · model_announcementArtificial Analysis
KAT-Coder-Pro V10.09582156732403314Not recordedNot recorded2025-11-11 · model_announcementArtificial Analysis
Claude 4.5 Haiku (Reasoning)0.09553946275772039Not recordedNot recorded2025-10-15 · model_announcementArtificial Analysis
Inkling Small0.09395052631730252Not recordedNot recorded2026-07-30 · model_announcementArtificial Analysis
Grok 4.3 (high)0.08118872318781814Not recordedNot recorded2026-04-30 · model_announcementArtificial Analysis
Step 3.7 Flash0.0700460712463557Not recordedNot recorded2026-05-29 · model_announcementArtificial Analysis
Nemotron 3 Ultra 550B A55B (Reasoning)0.05670562350908316Not recordedNot recorded2026-06-04 · model_announcementArtificial Analysis
Ring-2.6-1T0.021013617465460616Not recordedNot recorded2026-05-08 · model_announcementArtificial Analysis
GPT-5.5 Instant (June 2026)0.013565301670673362Not recordedNot recorded2026-06-25 · model_announcementArtificial Analysis

Put this benchmark in context

Tools and agents benchmarks