Six Navers Lab / Einsia benchmarks and the work they test
Compare the tasks, scoring and evidence behind six Einsia benchmarks, with complete Radar detail pages and original references.
Model reports
Brain-to-Text Benchmark '24
Evaluates 1,200 held-out sentences from one participant (T12). Language model post-processing is part of each system, and results do not establish transfer to another participant or to the different Brain-to-Text '25 dataset.
Evidence available for this benchmark
This source record contains 4 numeric observations. The source distinguishes 4 model identities or configurations. A model may appear more than once; check the version, tools and run budget before comparing. Read results and row-level evidence →
- Reported metric
- word_error_rate
- Recorded unit
- percent
- Metric direction
- lower is better
Start with these resources
Cited documents and dates (1)
Evaluation context and evidence
- Source
- Model reports
- Publisher
- Not recorded
- Benchmark release
- 2023-06-01
- Collected
- Not recorded
- Modality
- Not recorded
These are source-reported observations, not independent reruns by this site. Check the task version, evaluation protocol, unit and reporting date before comparing models. Results from different setups are not a universal ranking.
Reported results 4
Shown in source observation order, not ranked. Dates retain the precision provided by the source.
| Model | Value | Unit | Protocol | Reported date and basis | Source |
|---|---|---|---|---|---|
| DCoND-LIFT | 5.81 | percent | 1,200 competitionHoldOut sentences; aggregate word error rate | 2024-06-01 · document_publication | Brain-to-Text Benchmark '24 leaderboard |
| PyTorch baseline | 9.76 | percent | 1,200 competitionHoldOut sentences; aggregate word error rate | 2024-06-01 · document_publication | Brain-to-Text Benchmark '24 leaderboard |
| TeamCyber submission | 8.26 | percent | 1,200 competitionHoldOut sentences; aggregate word error rate | 2024-06-01 · document_publication | Brain-to-Text Benchmark '24 leaderboard |
| Tyler Benster et al. submission | 8.93 | percent | 1,200 competitionHoldOut sentences; aggregate word error rate | 2024-06-01 · document_publication | Brain-to-Text Benchmark '24 leaderboard |