Six new benchmark pages: context, agents and executable tasks
Explore six newly added benchmark detail pages, with task explanations, reported results and original publications.
These six benchmarks ask practical questions about reasoning, personalization, visual detail, negotiation, retrieval and software migration. Their papers were submitted on October 29–30; November 2 is the date these Radar pages were added. Start with the work you want to evaluate, then open the full methods and results.
Choose the work you need to test
Each detail page keeps its own metric, evaluation setting and source. The links below open the full benchmark records; the scores do not form a combined ranking.
| Benchmark | Task | Read with the result |
|---|---|---|
| GraphNIAH | Long-context graph reasoning | Accuracy by question and graph setting |
| P-ContextBench | Personal context and action boundaries | Capability breakdown and run consistency |
| CompoSET | Controlled visual compositionality | Matching protocol and edit category |
| TradeWar | Execution-grounded negotiation | Settlement outcomes and adaptation setting |
| ProSearch | Scientific protocol retrieval | nDCG@10 and retrieval pipeline |
| ROSTrans | Robotics software migration | Runtime success and repair budget |
- GraphNIAHModel reports
- P-ContextBenchModel reports
- CompoSETModel reports
- TradeWarModel reports
- ProSearchModel reports
- ROSTransModel reports
GraphNIAH · original paper and version history · P-ContextBench · original paper and version history · CompoSET · original paper and version history · TradeWar · original paper and version history · ProSearch · original paper and version history · ROSTrans · original paper and version history
GraphNIAH: Can a model follow a chain of references through a long context?
Uses scientific citation graphs to test reasoning when all required facts are in the supplied context. Read results by question type, graph size and path depth; finding one isolated fact is a different task.
- GraphNIAHModel reports
P-ContextBench: Can a personal agent use context without ignoring a person’s constraints?
Distributes personal facts across evolving synthetic app records. The tasks cover fact recovery, changing states, routine patterns and requests that conflict with a persona’s constraints. Read capability-specific results and consistency across repeated runs alongside overall accuracy.
- P-ContextBenchModel reports
CompoSET: Does the model notice the one detail that changes the meaning?
Pairs visually similar scenes that differ by one controlled edit, such as a color, count or spatial relation. This limits the scene-level shortcuts that can make image–text matching look easier. Its reported experiments concern dual-encoder vision-language models.
- CompoSETModel reports
TradeWar: Does a negotiated agreement still work when it is executed?
Turns negotiation into tariffs and trade allocations inside a multi-country simulation. Capacity and network constraints affect settlement, so agreement alone does not establish a good outcome. Read the Economic Health Index together with agent memory, adaptation and shock conditions.
- TradeWarModel reports
ProSearch: Can retrieval satisfy several experimental constraints at once?
Retrieves experimental protocols from queries containing several linked requirements, including numbers and exclusions. The benchmark spans six experimental domains. Keep first-stage retrieval and reranking configurations separate when reading nDCG@10.
- ProSearchModel reports
ROSTrans: Does migrated robotics code work after it compiles?
Asks models to migrate ROS1 code into fixed ROS2 packages, then launches nodes and exercises their interfaces. The protected reference tests check runtime behavior. Initial generation, multiple-sample pass rates and feedback-guided repair measure different conditions.
- ROSTransModel reports