AI EVALUATION, IN CONTEXT.BY EQUALITY MACHINE
Benchmark Radar.Benchmarks. Model results. Original sources.
/

Model reports

AI4AI-Bench

编程 Agent 能否改进现有 AI 研究项目中的算法？AI4AI-Bench 让它修改可运行的研究仓库，再用固定预算从头执行提交的源码，检验改进是否成立。

官方项目原论文官方仓库收录来源

AI Benchmark Radar / Equality Machine · 核对于 2026-10-27

对应 AI4AI-Bench 的十任务套件。论文 v1 成绩与官网标注 v1.5 的图表数据分别保留；模型、Agent 框架和推理强度共同定义一次配置。

十种算法家族

十个固定项目覆盖监督微调、Agent 强化学习、同策略蒸馏、奖励建模、偏好优化、扩散强化学习、遗忘学习、图扩散、模型融合和剪枝。起始模型与资产固定，Agent 可以修改学习方法和训练代码。Model Soup 与 OWL 不进行训练，分别组合已有权重和执行剪枝。

AI4AI-Bench paper v1 — task protocol, scoring and Tables 2–3 · Official AI4AI-Bench repository

探索与正式执行分开

每个配置有一张 NVIDIA B300 和四小时探索时间，可以读代码并用代理指标试验。只有源码进入全新的正式环境。八个训练任务从初始化状态运行，最多十二小时，并从最近至多三个检查点中取最佳结果；两个非训练任务执行一次，十二小时只是验收上限。

AI4AI-Bench paper v1 — task protocol, scoring and Tables 2–3

怎样读归一化总分

各任务的原始指标先映射到 0–1，再取平均：0 对应无信息模型，0.1 对应同条件下未经修改的原仓库方法，1 对应指标定义的理想点。困惑度使用对数坐标。0.288 不等于完成了 28.8% 的任务。论文对未交付模型的提交计归一化零分；原始执行凭据中的缺失或无效结果，不能强行改成任务原始指标的零。

AI4AI-Bench paper v1 — task protocol, scoring and Tables 2–3 · Official evaluation and replay documentation

比较配置，不混淆汇总口径

研究包含六个系统的 29 个配置，每个配置都尝试十道任务。论文中 Claude Opus 5 的 medium 配置为 0.288，而 0.250 是它五档推理强度的平均，不能混为同一成绩。官网 JSON 自标 v1.5，并保留更多小数，例如 medium 为 0.2881；应保留各自来源及原始精度，不互相覆盖。

AI4AI-Bench paper v1 — task protocol, scoring and Tables 2–3 · Official chart data — AI4AI-Bench v1.5 mapped scores

结果的适用边界

它测量单卡和固定时间内对十个特定项目的改进，并未演示完整的递归自我改进循环。基线是仓库原有方法，不是人类专家。部分代理指标与最终评测来自同一语料，因此分开执行不代表样本完全隔离。公开版本支持自托管评测，文档明确尚无官方盲测服务；使用默认警告模式的本地结果不算官方结果。

AI4AI-Bench paper v1 — task protocol, scoring and Tables 2–3 · Official evaluation and replay documentation

官网图表中的全部 29 个配置

下表逐项保留官网 points.json 的平均值，源文件标注“AI4AI-Bench v1.5 mapped scores”，于 2026 年 10 月 27 日核对。每个配置都尝试十道任务。文件未给出精确发布日期，因此核对日期不作为新版本发布日期或成绩报告日期。

官网 v1.5 图表数据 · 全部十道任务 · 2026 年 10 月 27 日核对
模型模型机构运行框架推理强度平均归一化分数
Claude Opus 5AnthropicClaude Codelow0.2722
Claude Opus 5AnthropicClaude Codemedium0.2881
Claude Opus 5AnthropicClaude Codehigh0.2776
Claude Opus 5AnthropicClaude Codexhigh0.1898
Claude Opus 5AnthropicClaude Codemax0.2245
Claude Sonnet 5AnthropicClaude Codelow0.0991
Claude Sonnet 5AnthropicClaude Codemedium0.1092
Claude Sonnet 5AnthropicClaude Codehigh0.216
Claude Sonnet 5AnthropicClaude Codexhigh0.137
Claude Sonnet 5AnthropicClaude Codemax0.1635
Kimi K3Moonshot AIClaude Codemax0.1738
GPT-5.6 SolOpenAICodexnone0.1261
GPT-5.6 SolOpenAICodexlow0.1976
GPT-5.6 SolOpenAICodexmedium0.144
GPT-5.6 SolOpenAICodexhigh0.235
GPT-5.6 SolOpenAICodexxhigh0.1952
GPT-5.6 SolOpenAICodexmax0.2455
GPT-5.6 TerraOpenAICodexnone0.065
GPT-5.6 TerraOpenAICodexlow0.1085
GPT-5.6 TerraOpenAICodexmedium0.1349
GPT-5.6 TerraOpenAICodexhigh0.1293
GPT-5.6 TerraOpenAICodexxhigh0.1438
GPT-5.6 TerraOpenAICodexmax0.2282
GPT-5.6 LunaOpenAICodexnone0.0914
GPT-5.6 LunaOpenAICodexlow0.0786
GPT-5.6 LunaOpenAICodexmedium0.1242
GPT-5.6 LunaOpenAICodexhigh0.1191
GPT-5.6 LunaOpenAICodexxhigh0.1468
GPT-5.6 LunaOpenAICodexmax0.1393

保留 JSON 原始精度。量表为 0–1，原仓库方法基线为 0.1；这些数字不是百分比。成绩对应模型、框架与推理强度的组合。论文表 3 保留其独立的较低精度读数，本表不覆盖论文成绩。

Official chart data — AI4AI-Bench v1.5 mapped scores · AI4AI-Bench official project and results · AI4AI-Bench paper v1 — task protocol, scoring and Tables 2–3

六个系统在已测推理强度上的汇总

官方文件还给出以下六个系统汇总。每行合并所列的推理强度，因此分母不同：Claude 测五档，GPT-5.6 测六档，Kimi K3 只测一档。它们不是新增的单一推理强度配置。

官方跨配置平均值 · 单元数是配置×任务，不是独立题目数
模型运行框架纳入的推理强度配置×任务单元数汇总平均分数
Claude Opus 5Claude Codelow, medium, high, xhigh, max500.25
GPT-5.6 SolCodexnone, low, medium, high, xhigh, max600.191
Kimi K3Claude Codemax100.174
Claude Sonnet 5Claude Codelow, medium, high, xhigh, max500.145
GPT-5.6 TerraCodexnone, low, medium, high, xhigh, max600.135
GPT-5.6 LunaCodexnone, low, medium, high, xhigh, max600.117

各行使用同一套十道任务，但合并的配置数量不同。保留作者发布的原始均值，不用它们替换上表 29 个单独配置的分数。

Official chart data — AI4AI-Bench v1.5 mapped scores · AI4AI-Bench paper v1 — task protocol, scoring and Tables 2–3

原始发布

对应 AI4AI-Bench 的十任务套件。论文 v1 成绩与官网标注 v1.5 的图表数据分别保留；模型、Agent 框架和推理强度共同定义一次配置。

核对于 2026-10-27

已收录的评测资料

这条来源记录收录了 29 条数值成绩。来源区分了 6 个模型身份或配置。同一模型可能有多条记录；比较前请核对具体版本、工具和运行预算。 查看成绩及逐条证据 →

报告指标
Mean normalized score
原始单位
points
指标方向
数值越高越好

从这些资料开始

引用文档及日期（1）

关注度快照

观测日期：2026-10-11

Hugging Face 论文点赞
3

查看来源快照

这是注明日期的关注度观测，不是实时计数或评测分数。

评测信息与证据

来源
Model reports
发布者
未收录
评测发布日期
2026-08-20
采集日期
未收录
模态
未收录

本站保留来源报告的结果，不代表独立复测。请先核对任务版本、评测协议、分数单位和报告日期；不同运行条件不能直接合并为统一排名。

报告成绩 29

按来源中的观察顺序展示，顺序不代表排名。日期保留原始精度。

模型数值单位协议报告日期与口径来源
Claude Opus 50.278pointsClaude Code; reasoning effort high; 10 tasks; 4 h exploration on 1 NVIDIA B300; 8 training tasks: clean-start run up to 12 h, best of up to 3 latest checkpoints; 2 non-training tasks: one execution under 12 h verification ceiling; mean mapped score (paper v1 Table 3)2026-08-20 · document_publicationAI4AI-Bench paper v1 — mean mapped scores, Table 3
Claude Sonnet 50.216pointsClaude Code; reasoning effort high; 10 tasks; 4 h exploration on 1 NVIDIA B300; 8 training tasks: clean-start run up to 12 h, best of up to 3 latest checkpoints; 2 non-training tasks: one execution under 12 h verification ceiling; mean mapped score (paper v1 Table 3)2026-08-20 · document_publicationAI4AI-Bench paper v1 — mean mapped scores, Table 3
Claude Opus 50.272pointsClaude Code; reasoning effort low; 10 tasks; 4 h exploration on 1 NVIDIA B300; 8 training tasks: clean-start run up to 12 h, best of up to 3 latest checkpoints; 2 non-training tasks: one execution under 12 h verification ceiling; mean mapped score (paper v1 Table 3)2026-08-20 · document_publicationAI4AI-Bench paper v1 — mean mapped scores, Table 3
Claude Sonnet 50.099pointsClaude Code; reasoning effort low; 10 tasks; 4 h exploration on 1 NVIDIA B300; 8 training tasks: clean-start run up to 12 h, best of up to 3 latest checkpoints; 2 non-training tasks: one execution under 12 h verification ceiling; mean mapped score (paper v1 Table 3)2026-08-20 · document_publicationAI4AI-Bench paper v1 — mean mapped scores, Table 3
Claude Opus 50.225pointsClaude Code; reasoning effort max; 10 tasks; 4 h exploration on 1 NVIDIA B300; 8 training tasks: clean-start run up to 12 h, best of up to 3 latest checkpoints; 2 non-training tasks: one execution under 12 h verification ceiling; mean mapped score (paper v1 Table 3)2026-08-20 · document_publicationAI4AI-Bench paper v1 — mean mapped scores, Table 3
Claude Sonnet 50.163pointsClaude Code; reasoning effort max; 10 tasks; 4 h exploration on 1 NVIDIA B300; 8 training tasks: clean-start run up to 12 h, best of up to 3 latest checkpoints; 2 non-training tasks: one execution under 12 h verification ceiling; mean mapped score (paper v1 Table 3)2026-08-20 · document_publicationAI4AI-Bench paper v1 — mean mapped scores, Table 3
Kimi K30.174pointsClaude Code; reasoning effort max; 10 tasks; 4 h exploration on 1 NVIDIA B300; 8 training tasks: clean-start run up to 12 h, best of up to 3 latest checkpoints; 2 non-training tasks: one execution under 12 h verification ceiling; mean mapped score (paper v1 Table 3)2026-08-20 · document_publicationAI4AI-Bench paper v1 — mean mapped scores, Table 3
Claude Opus 50.288pointsClaude Code; reasoning effort medium; 10 tasks; 4 h exploration on 1 NVIDIA B300; 8 training tasks: clean-start run up to 12 h, best of up to 3 latest checkpoints; 2 non-training tasks: one execution under 12 h verification ceiling; mean mapped score (paper v1 Table 3)2026-08-20 · document_publicationAI4AI-Bench paper v1 — mean mapped scores, Table 3
Claude Sonnet 50.109pointsClaude Code; reasoning effort medium; 10 tasks; 4 h exploration on 1 NVIDIA B300; 8 training tasks: clean-start run up to 12 h, best of up to 3 latest checkpoints; 2 non-training tasks: one execution under 12 h verification ceiling; mean mapped score (paper v1 Table 3)2026-08-20 · document_publicationAI4AI-Bench paper v1 — mean mapped scores, Table 3
Claude Opus 50.19pointsClaude Code; reasoning effort xhigh; 10 tasks; 4 h exploration on 1 NVIDIA B300; 8 training tasks: clean-start run up to 12 h, best of up to 3 latest checkpoints; 2 non-training tasks: one execution under 12 h verification ceiling; mean mapped score (paper v1 Table 3)2026-08-20 · document_publicationAI4AI-Bench paper v1 — mean mapped scores, Table 3
Claude Sonnet 50.137pointsClaude Code; reasoning effort xhigh; 10 tasks; 4 h exploration on 1 NVIDIA B300; 8 training tasks: clean-start run up to 12 h, best of up to 3 latest checkpoints; 2 non-training tasks: one execution under 12 h verification ceiling; mean mapped score (paper v1 Table 3)2026-08-20 · document_publicationAI4AI-Bench paper v1 — mean mapped scores, Table 3
GPT-5.6 Luna0.119pointsCodex; reasoning effort high; 10 tasks; 4 h exploration on 1 NVIDIA B300; 8 training tasks: clean-start run up to 12 h, best of up to 3 latest checkpoints; 2 non-training tasks: one execution under 12 h verification ceiling; mean mapped score (paper v1 Table 3)2026-08-20 · document_publicationAI4AI-Bench paper v1 — mean mapped scores, Table 3
GPT-5.6 Sol0.235pointsCodex; reasoning effort high; 10 tasks; 4 h exploration on 1 NVIDIA B300; 8 training tasks: clean-start run up to 12 h, best of up to 3 latest checkpoints; 2 non-training tasks: one execution under 12 h verification ceiling; mean mapped score (paper v1 Table 3)2026-08-20 · document_publicationAI4AI-Bench paper v1 — mean mapped scores, Table 3
GPT-5.6 Terra0.129pointsCodex; reasoning effort high; 10 tasks; 4 h exploration on 1 NVIDIA B300; 8 training tasks: clean-start run up to 12 h, best of up to 3 latest checkpoints; 2 non-training tasks: one execution under 12 h verification ceiling; mean mapped score (paper v1 Table 3)2026-08-20 · document_publicationAI4AI-Bench paper v1 — mean mapped scores, Table 3
GPT-5.6 Luna0.079pointsCodex; reasoning effort low; 10 tasks; 4 h exploration on 1 NVIDIA B300; 8 training tasks: clean-start run up to 12 h, best of up to 3 latest checkpoints; 2 non-training tasks: one execution under 12 h verification ceiling; mean mapped score (paper v1 Table 3)2026-08-20 · document_publicationAI4AI-Bench paper v1 — mean mapped scores, Table 3
GPT-5.6 Sol0.198pointsCodex; reasoning effort low; 10 tasks; 4 h exploration on 1 NVIDIA B300; 8 training tasks: clean-start run up to 12 h, best of up to 3 latest checkpoints; 2 non-training tasks: one execution under 12 h verification ceiling; mean mapped score (paper v1 Table 3)2026-08-20 · document_publicationAI4AI-Bench paper v1 — mean mapped scores, Table 3
GPT-5.6 Terra0.109pointsCodex; reasoning effort low; 10 tasks; 4 h exploration on 1 NVIDIA B300; 8 training tasks: clean-start run up to 12 h, best of up to 3 latest checkpoints; 2 non-training tasks: one execution under 12 h verification ceiling; mean mapped score (paper v1 Table 3)2026-08-20 · document_publicationAI4AI-Bench paper v1 — mean mapped scores, Table 3
GPT-5.6 Luna0.139pointsCodex; reasoning effort max; 10 tasks; 4 h exploration on 1 NVIDIA B300; 8 training tasks: clean-start run up to 12 h, best of up to 3 latest checkpoints; 2 non-training tasks: one execution under 12 h verification ceiling; mean mapped score (paper v1 Table 3)2026-08-20 · document_publicationAI4AI-Bench paper v1 — mean mapped scores, Table 3
GPT-5.6 Sol0.245pointsCodex; reasoning effort max; 10 tasks; 4 h exploration on 1 NVIDIA B300; 8 training tasks: clean-start run up to 12 h, best of up to 3 latest checkpoints; 2 non-training tasks: one execution under 12 h verification ceiling; mean mapped score (paper v1 Table 3)2026-08-20 · document_publicationAI4AI-Bench paper v1 — mean mapped scores, Table 3
GPT-5.6 Terra0.228pointsCodex; reasoning effort max; 10 tasks; 4 h exploration on 1 NVIDIA B300; 8 training tasks: clean-start run up to 12 h, best of up to 3 latest checkpoints; 2 non-training tasks: one execution under 12 h verification ceiling; mean mapped score (paper v1 Table 3)2026-08-20 · document_publicationAI4AI-Bench paper v1 — mean mapped scores, Table 3
GPT-5.6 Luna0.124pointsCodex; reasoning effort medium; 10 tasks; 4 h exploration on 1 NVIDIA B300; 8 training tasks: clean-start run up to 12 h, best of up to 3 latest checkpoints; 2 non-training tasks: one execution under 12 h verification ceiling; mean mapped score (paper v1 Table 3)2026-08-20 · document_publicationAI4AI-Bench paper v1 — mean mapped scores, Table 3
GPT-5.6 Sol0.144pointsCodex; reasoning effort medium; 10 tasks; 4 h exploration on 1 NVIDIA B300; 8 training tasks: clean-start run up to 12 h, best of up to 3 latest checkpoints; 2 non-training tasks: one execution under 12 h verification ceiling; mean mapped score (paper v1 Table 3)2026-08-20 · document_publicationAI4AI-Bench paper v1 — mean mapped scores, Table 3
GPT-5.6 Terra0.135pointsCodex; reasoning effort medium; 10 tasks; 4 h exploration on 1 NVIDIA B300; 8 training tasks: clean-start run up to 12 h, best of up to 3 latest checkpoints; 2 non-training tasks: one execution under 12 h verification ceiling; mean mapped score (paper v1 Table 3)2026-08-20 · document_publicationAI4AI-Bench paper v1 — mean mapped scores, Table 3
GPT-5.6 Luna0.091pointsCodex; reasoning effort none; 10 tasks; 4 h exploration on 1 NVIDIA B300; 8 training tasks: clean-start run up to 12 h, best of up to 3 latest checkpoints; 2 non-training tasks: one execution under 12 h verification ceiling; mean mapped score (paper v1 Table 3)2026-08-20 · document_publicationAI4AI-Bench paper v1 — mean mapped scores, Table 3
GPT-5.6 Sol0.126pointsCodex; reasoning effort none; 10 tasks; 4 h exploration on 1 NVIDIA B300; 8 training tasks: clean-start run up to 12 h, best of up to 3 latest checkpoints; 2 non-training tasks: one execution under 12 h verification ceiling; mean mapped score (paper v1 Table 3)2026-08-20 · document_publicationAI4AI-Bench paper v1 — mean mapped scores, Table 3
GPT-5.6 Terra0.065pointsCodex; reasoning effort none; 10 tasks; 4 h exploration on 1 NVIDIA B300; 8 training tasks: clean-start run up to 12 h, best of up to 3 latest checkpoints; 2 non-training tasks: one execution under 12 h verification ceiling; mean mapped score (paper v1 Table 3)2026-08-20 · document_publicationAI4AI-Bench paper v1 — mean mapped scores, Table 3
GPT-5.6 Luna0.147pointsCodex; reasoning effort xhigh; 10 tasks; 4 h exploration on 1 NVIDIA B300; 8 training tasks: clean-start run up to 12 h, best of up to 3 latest checkpoints; 2 non-training tasks: one execution under 12 h verification ceiling; mean mapped score (paper v1 Table 3)2026-08-20 · document_publicationAI4AI-Bench paper v1 — mean mapped scores, Table 3
GPT-5.6 Sol0.195pointsCodex; reasoning effort xhigh; 10 tasks; 4 h exploration on 1 NVIDIA B300; 8 training tasks: clean-start run up to 12 h, best of up to 3 latest checkpoints; 2 non-training tasks: one execution under 12 h verification ceiling; mean mapped score (paper v1 Table 3)2026-08-20 · document_publicationAI4AI-Bench paper v1 — mean mapped scores, Table 3
GPT-5.6 Terra0.144pointsCodex; reasoning effort xhigh; 10 tasks; 4 h exploration on 1 NVIDIA B300; 8 training tasks: clean-start run up to 12 h, best of up to 3 latest checkpoints; 2 non-training tasks: one execution under 12 h verification ceiling; mean mapped score (paper v1 Table 3)2026-08-20 · document_publicationAI4AI-Bench paper v1 — mean mapped scores, Table 3

相关评测探索

按共享资料或来源分类推荐，不代表同一评测或经审核的身份关联。

PresentBench来源分类相同 →Kimi Code Bench v2来源分类相同 →SpreadsheetBench 2来源分类相同 →CoWorkBench来源分类相同 →OSWorld-Verified来源分类相同 →BrowseComp来源分类相同 →LiveSQLBench来源分类相同 →Siren AgentDojo Utility来源分类相同 →

继续阅读

编程评测工具与智能体评测

继续探索

按这条记录的来源标签检索目录；搜索匹配不代表相同评测。

agents →Model reports · 全部记录 →