Six Navers Lab / Einsia benchmarks and the work they test
Compare the tasks, scoring and evidence behind six Einsia benchmarks, with complete Radar detail pages and original references.
The research index mixes benchmarks, methods and tools. This guide separates six evaluation projects and shows which kind of work each one can help you inspect.
Six benchmarks, six different jobs
Start with the work an agent must deliver. These projects cover engineering optimization, learning algorithms, repository migration, editable diagrams, video physics and quantitative research. Their scores measure different outcomes and should be read with each project’s task definition.
| Benchmark | What it tests | How to read the result |
|---|---|---|
| Frontier-Engineering | Engineering optimization under executable constraints | Corrected medal score; v1 and v1-lite separate |
| AI4AI-Bench | Improve the training method inside a frozen research codebase | Read task normalization and protocol version |
| SWE Refactor Bench | Migrate a whole repository while preserving behavior | Composite score and full acceptance differ |
| PPTBench (Einsia) | Reconstruct scientific flow diagrams as editable slides | Artifact and semantic gates precede visual scoring |
| World Models’ Last Exam in Physics | Keep physical behavior consistent in generated video | Read task scoring and sampling conditions |
| Quant-Arena | Research and freeze a strategy before sealed-year evaluation | Total score is not a financial return |
- Frontier-EngineeringModel reports
- AI4AI-BenchModel reports
- SWE Refactor BenchModel reports
- PPTBench (Einsia)Model reports
- World Models’ Last Exam in PhysicsModel reports
- Quant-ArenaModel reports
Executable work: optimization, algorithms and migration
Frontier-Engineering judges how a candidate improves under domain-specific verifiers. AI4AI-Bench asks an agent to modify a research codebase, with training and evaluation constraints. SWE Refactor Bench starts from working repositories and checks whether a migration keeps their behavior. A high score on one does not establish the others.
Frontier’s corrected medal results combine rescored submissions and shorter reruns. Historical average ranks answer a different question. The detail pages retain versions and original sources, so the update is not mistaken for a uniform new experiment.
- Frontier-EngineeringModel reports
- AI4AI-BenchModel reports
- SWE Refactor BenchModel reports
Frontier-Engineering · official project · AI4AI-Bench · official project · SWE Refactor Bench · official project · Frontier-Engineering · corrected-result protocol
Look beyond a plausible picture
Einsia’s PPTBench is the scientific-flow-diagram reconstruction benchmark with 500 tasks, rather than every earlier project sharing the name. The output must remain editable as native PowerPoint elements, and diagram semantics matter alongside appearance.
World Models’ Last Exam in Physics evaluates generated videos with physical tasks and task-specific measurements. The research index labels it Research Preview, and its sampling conditions remain part of the evidence; a visually convincing video alone is not proof that its physics is right.
- PPTBench (Einsia)Model reports
- World Models’ Last Exam in PhysicsModel reports
PPTBench (Einsia) · official project · World Models’ Last Exam in Physics · official project
Separate a research score from portfolio return
Quant-Arena separates a research window from a sealed assessment year. Its total combines research-period and sealed-period scores. Both iterative and one-shot settings, and every reasoning effort, are retained. The publisher’s proprietary Kepler-1 reference is labeled separately.
The detail explains a useful distinction: selecting each model’s highest total score is not the same as selecting its highest annualized return. Historical results under these trading assumptions do not establish future investment performance.
- Quant-ArenaModel reports
References
- Navers Lab / Einsia · Research index
- Frontier-Engineering · official project
- AI4AI-Bench · official project
- SWE Refactor Bench · official project
- PPTBench (Einsia) · official project
- World Models’ Last Exam in Physics · official project
- Quant-Arena · official project
- EmbodiedEvo · official project
- BrowserBC · official project
- OpenChronicle · official project
- Frontier-Engineering · corrected-result protocol