Six Navers Lab / Einsia benchmarks and the work they test
Compare the tasks, scoring and evidence behind six Einsia benchmarks, with complete Radar detail pages and original references.
Model reports
SWE-Lancer
Software engineering freelancer tasks from Upwork; real-world constraints and client communication are part of the task.
Evidence available for this benchmark
This source record contains 2 numeric observations. The source distinguishes 2 model identities or configurations. A model may appear more than once; check the version, tools and run budget before comparing. Read results and row-level evidence →
- Reported metric
- resolved
- Recorded unit
- percent
- Metric direction
- higher is better
Start with these resources
Cited documents and dates (1)
Evaluation context and evidence
- Source
- Model reports
- Publisher
- Not recorded
- Benchmark release
- 2025-02-17
- Collected
- Not recorded
- Modality
- Not recorded
These are source-reported observations, not independent reruns by this site. Check the task version, evaluation protocol, unit and reporting date before comparing models. Results from different setups are not a universal ranking.
Reported results 2
Shown in source observation order, not ranked. Dates retain the precision provided by the source.
| Model | Value | Unit | Protocol | Reported date and basis | Source |
|---|---|---|---|---|---|
| Seed2.0 Pro | 49.4 | percent | as printed in Table 11 (Large Models) | 2026-02-14 · document_publication | Seed2.0 (Pro / Lite / Mini) |
| Seed2.0 Lite | 47.1 | percent | as printed in Table 12 (Efficient Models) | 2026-02-14 · document_publication | Seed2.0 (Pro / Lite / Mini) |