Four new benchmark pages: teamwork, rule compliance and visual agents
Explore Collaborative Intelligence, ARCBench, Shotwell-Bench and LOLPlay with task explanations, original publications and versioned reported results.
Find evaluations for agent teamwork, rules after context compaction, video action annotation and real-time game control. Follow each detail page to inspect its source, metric and experimental settings.
Choose the behavior you want to evaluate
Open each detail page for its task explanation, versioned results and original publication links. These four evaluations measure different behaviors; the table points to the conditions that help you interpret their scores.
| Benchmark | Task | Read with the score |
|---|---|---|
| Collaborative Intelligence Benchmark | Working as a teammate | Eligible tasks, eight dimensions and reasoning setting |
| ARCBench | Rule compliance after context compaction | Violation rate, repetition count and compaction protocol |
| Shotwell-Bench | Video action segmentation and description | Two F1 components, input format and judge |
| LOLPlay | Real-time visual game control | Full tasks versus subgoals, effort and time budget |
- Collaborative Intelligence BenchmarkModel reports
- ARCBenchModel reports
- Shotwell-BenchModel reports
- LOLPlayModel reports
Collaborative Intelligence Benchmark · original publication · ARCBench · original publication · Shotwell-Bench · original publication · LOLPlay · original publication
Collaborative Intelligence Benchmark: Can an agent cooperate on tasks it can already solve alone?
The benchmark tests eight teamwork dimensions across 240 tasks. Its score is conditioned on what each model can solve alone, so models do not necessarily face the same eligible subset. The detail page keeps default and high reasoning settings separate and links the original task and scoring materials.
- Collaborative Intelligence BenchmarkModel reports
ARCBench: Does retaining a rule mean the agent will obey it?
The primary comparison measures whether any of three valid repetitions violates a rule. Full history and compacted history are different protocols, and a lower violation rate is better. Supplementary tables separately show rule delivery, compaction methods, budgets and adversarial conditions.
- ARCBenchModel reports
Shotwell-Bench: Can a video model identify action boundaries and describe the action?
The current comparison covers eleven selected videos; an earlier five-video pilot is retained separately. Overall F1 multiplies segmentation and description F1. Native-video and contact-sheet inputs differ, and the description judge is also one of the evaluated models, so the component scores and settings matter.
- Shotwell-BenchModel reports
LOLPlay: Can an agent act in time while the game keeps moving?
Thirty-six champion-specific League of Legends tasks are evaluated from screenshots and mouse–keyboard actions. Nine model–effort configurations have full-task success rates and separate partial-progress results. These are bounded tasks with executable checks, not full-match win rates or results against human opponents.
- LOLPlayModel reports
New Radar pages, with source dates preserved
November 4 is the date these pages were added and reviewed. Collaborative Intelligence and ARCBench retain their November 2 paper dates. Shotwell results cite dated author repository revisions; its experiment dates are not inferred. LOLPlay retains the October 25 submission date displayed in the original arXiv record. Radar transcribed the reported results and did not rerun these evaluations.
Collaborative Intelligence Benchmark · original publication · ARCBench · original publication · Shotwell-Bench · original publication · LOLPlay · original publication