Model reports

WorkIng

Detect obstacles and plan a path through highway roadworks. Compare real driving with simulation, then inspect whether synthetic training actually transfers to the real test.

Original paper Official repository Catalog source

AI Benchmark Radar / Equality Machine · Reviewed 2026-11-03

Original paper v1, submitted 30 October 2026; reviewed and added 3 November. Primary scores use the fixed real daytime test; other scenes and tasks are separate tables.

What changes in a work zone?

Temporary lanes, barriers and signage challenge perception and planning. WorkIng-Highway combines six cameras, 128-channel LiDAR and RTK-GPS; WorkIng-CARLA supplies procedural work-zone scenarios.

WorkIng original paper v1

Keep scenes and splits separate

Real day/night sequences use non-overlapping sub-sequences and 80/10/10 train/validation/test splits. CARLA holds out a town. Night omits sparse classes; synthetic results therefore do not use the same evaluation population.

WorkIng original paper v1 · WorkIng · Table 4

The primary score has one scope

The primary series uses Table 5 daytime NDS for all 15 detector/training configurations. NDS is printed on a 0–100 scale; this variant omits AAE. Night, CARLA, detection errors and planning outcomes remain separate.

WorkIng original paper v1 · WorkIng · Table 5

Read the training data labels

W means WorkIng-Highway, D DrivIng, and C WorkIng-CARLA. W+D+C uses scenario-stratified synthetic sampling. Training only in simulation is distinct from testing in simulation.

WorkIng · Table 5 · WorkIng original paper v1

Publication versus runnable resources

The paper is accessible. On 3 November the author GitHub repository contains README and LICENSE only; it does not yet verify availability of the full dataset, runners or reproduction configs.

WorkIng · author repository (release placeholder) · WorkIng original paper v1

Table 4 · within-domain detection

Training and testing each use the named scene.

Table 4 · within-domain detection
Scene	Model	ATE ↓	ASE ↓	AOE ↓	mAP ↑	NDS ↑
Day	CenterPoint	0.21	0.18	0.07	0.76	79.40
Day	PETR	0.87	0.27	0.24	0.37	43.52
Day	BEVFusion	0.23	0.23	0.14	0.75	76.59
Night	CenterPoint	0.14	0.15	0.07	0.78	81.55
Night	PETR	0.87	0.18	0.06	0.33	44.04
Night	BEVFusion	0.22	0.25	0.20	0.78	77.71
CARLA	CenterPoint	0.05	0.15	0.06	0.97	94.65
CARLA	PETR	0.66	0.19	0.05	0.56	61.35
CARLA	BEVFusion	0.20	0.32	0.32	0.93	85.12

Day, Night and CARLA have different test populations. mAP is a decimal; NDS is x100.

WorkIng · Table 4

Table 5 · Day test by training source

All five training configurations use the same real test split within this scene.

Table 5 · Day test by training source
Model	Training	ATE ↓	ASE ↓	AOE ↓	mAP ↑	NDS ↑
CenterPoint	W	0.21	0.18	0.07	0.76	79.40
CenterPoint	D	0.22	0.24	0.31	0.45	55.81
CenterPoint	C	0.50	0.61	1.07	0.13	18.37
CenterPoint	W+D	0.14	0.18	0.18	0.78	79.70
CenterPoint	W+D+C	0.14	0.18	0.17	0.79	80.75
PETR	W	0.87	0.27	0.24	0.37	43.52
PETR	D	1.14	0.39	0.23	0.11	22.37
PETR	C	1.14	0.52	1.05	0.01	4.32
PETR	W+D	0.66	0.19	0.20	0.37	47.44
PETR	W+D+C	0.66	0.12	0.07	0.41	52.27
BEVFusion	W	0.23	0.23	0.14	0.75	76.59
BEVFusion	D	0.33	0.42	1.41	0.08	15.70
BEVFusion	C	0.78	0.73	0.64	0.14	19.05
BEVFusion	W+D	0.14	0.18	0.15	0.79	81.33
BEVFusion	W+D+C	0.12	0.13	0.08	0.83	84.98

Only Day NDS enters the primary series. W rows repeat Table 4 and are not counted twice.

WorkIng · Table 5

Table 5 · Night test by training source

All five training configurations use the same real test split within this scene.

Table 5 · Night test by training source
Model	Training	ATE ↓	ASE ↓	AOE ↓	mAP ↑	NDS ↑
CenterPoint	W	0.14	0.15	0.07	0.78	81.55
CenterPoint	D	0.60	0.43	0.30	0.45	48.90
CenterPoint	C	0.69	0.67	0.45	0.25	30.36
CenterPoint	W+D	0.14	0.14	0.05	0.83	85.53
CenterPoint	W+D+C	0.13	0.12	0.04	0.86	87.63
PETR	W	0.87	0.18	0.06	0.33	44.04
PETR	D	1.09	0.45	0.59	0.07	15.17
PETR	C	1.13	0.92	1.19	0.00	0.97
PETR	W+D	0.61	0.21	0.11	0.40	50.70
PETR	W+D+C	0.57	0.26	0.15	0.44	52.31
BEVFusion	W	0.22	0.25	0.20	0.78	77.71
BEVFusion	D	0.65	0.66	0.91	0.24	24.50
BEVFusion	C	0.76	0.70	0.59	0.19	23.82
BEVFusion	W+D	0.16	0.19	0.14	0.82	82.76
BEVFusion	W+D+C	0.15	0.14	0.11	0.88	87.69

Only Day NDS enters the primary series. W rows repeat Table 4 and are not counted twice.

WorkIng · Table 5

Table 6 · separate ROADWork transfer test

This is 2D mask recognition on ROADWork, not the WorkIng 3D primary metric.

Table 6 · separate ROADWork transfer test
Training / provenance	Model	AP	AP50	AP75	APS	APM	APL
Zero-shot foundation models	OpenSeeD	2.9	5.8	2.5	1.3	2.8	4.0
Zero-shot foundation models	Detic	4.2	6.3	4.6	2.1	5.5	6.0
Zero-shot foundation models	OMG-LLaVA	0.2	0.3	0.2	0.0	0.1	0.8
Train: C (WorkIng-CARLA only)	Mask R-CNN	4.13	5.07	4.23	2.74	3.48	5.12
Train: C (WorkIng-CARLA only)	Mask DINO	5.65	8.82	5.05	2.49	4.46	7.44
Train: R (ROADWork only)	Mask R-CNN	30.23	49.30	32.41	17.09	32.78	42.60
Train: R (ROADWork only)	Mask DINO	38.01	55.70	41.76	19.20	38.12	51.27
Train: C+R (combined)	Mask R-CNN	35.56	50.65	34.42	19.58	33.74	39.39
Train: C+R (combined)	Mask DINO	43.15	59.73	44.01	21.02	43.47	47.54

All values are mask AP (%). The three zero-shot rows are cited from ROADWork, not rerun by WorkIng; other rows use one identical recipe per architecture.

WorkIng · Table 6

Table 7 · logged real-world planning

The vehicle does not react to a new plan in logged replay. Both tiny planners use the authors’ camera-LiDAR adaptation without an HD-map branch.

Table 7 · logged real-world planning
Scene	Model	L2 0.5s (m)	L2 1s (m)	L2 2s (m)	L2 avg (m)	Collision 0.5s (%)	Collision 1s (%)	Collision 2s (%)	Collision avg (%)
Day	UniAD_tiny	0.23	0.55	1.45	0.74	0.11	0.33	5.97	2.14
Day	VAD_tiny	0.18	0.43	1.25	0.62	0.14	0.34	5.13	1.87
Night	UniAD_tiny	0.37	0.70	1.63	0.90	0.85	0.88	4.84	2.19
Night	VAD_tiny	0.35	0.86	1.60	0.94	0.85	0.88	4.76	2.16

Lower is better for both metrics. These are not reactive driving success rates.

WorkIng · Table 7 · WorkIng original paper v1

Table 8 · reactive CARLA planning

A single run over 85 routes records the authors’ closed-loop result. The paper does not treat the 0.02 Driving Score gap as a ranking.

Table 8 · reactive CARLA planning
Model	L2 0.5s (m)	L2 1s (m)	L2 2s (m)	L2 avg (m)	Collision 0.5s (%)	Collision 1s (%)	Collision 2s (%)	Collision avg (%)	DS ↑	RC ↑	IP ↑
UniAD_tiny	0.23	0.48	1.34	0.68	0.23	0.75	2.84	1.27	0.63	0.80	0.79
VAD_tiny	0.25	0.54	1.19	0.66	0.08	0.54	1.96	0.86	0.65	0.76	0.85

DS = Driving Score; RC = Route Completion; IP = Infraction Penalty. Models are trained only on WorkIng-CARLA.

WorkIng · Table 8

Table 9 · planning transfer to real data

Training W, C or W+C changes the data, while the real test set stays fixed.

Table 9 · planning transfer to real data
Training	UniAD day L2 (m)	UniAD night L2 (m)	UniAD day collision (%)	UniAD night collision (%)	VAD day L2 (m)	VAD night L2 (m)	VAD day collision (%)	VAD night collision (%)
W	0.74	0.90	2.14	2.19	0.62	0.94	1.87	2.16
C	7.57	8.13	6.98	5.51	8.32	9.33	6.78	5.82
W+C	0.66	0.84	1.17	1.65	0.53	0.83	1.09	1.78

Values average the 0.5/1.0/2.0-second horizons. Lower is better.

WorkIng · Table 9

Table 12 · per-class detection

The author table retains each class rather than replacing excluded classes with zero.

Table 12 · per-class detection
Scene	Model	Car	Truck	Bus	Trailer	Barrier	Motorcycle	Pedestrian	Total
Day	CenterPoint	0.95	0.89	0.37	0.87	0.86	0.68	0.71	0.76
Day	PETR	0.60	0.47	0.18	0.39	0.39	0.45	0.12	0.37
Day	BEVFusion	0.95	0.85	0.37	0.83	0.80	0.80	0.64	0.75
Night	CenterPoint	0.88	0.87	-	0.44	0.92	-	-	0.78
Night	PETR	0.43	0.57	-	0.01	0.30	-	-	0.33
Night	BEVFusion	0.93	0.85	-	0.42	0.90	-	-	0.78
CARLA	CenterPoint	0.99	0.98	0.99	-	0.88	0.98	0.98	0.97
CARLA	PETR	0.64	0.52	0.53	-	0.65	0.55	0.49	0.56
CARLA	BEVFusion	0.97	0.95	0.94	-	0.82	0.96	0.92	0.93

All entries are decimal mAP. Dashes are excluded/absent classes.

WorkIng · Table 12

Original publication

Original paper v1, submitted 30 October 2026; reviewed and added 3 November. Primary scores use the fixed real daytime test; other scenes and tasks are separate tables.

Reviewed 2026-11-03

Evidence available for this benchmark

This source record contains 15 numeric observations. The source distinguishes 3 model identities or configurations. A model may appear more than once; check the version, tools and run budget before comparing. Read results and row-level evidence →

Reported metric: WorkIng-Highway DAY test NDS
Recorded unit: points
Metric direction: higher is better

Start with these resources

Catalog source · arxiv.org

Cited documents and dates (1)

WorkIng paper v1 · reported benchmark results · 2026-10-30

Evaluation context and evidence

Source: Model reports
Publisher: Not recorded
Benchmark release: 2026-10-30
Collected: Not recorded
Modality: Not recorded

These are source-reported observations, not independent reruns by this site. Check the task version, evaluation protocol, unit and reporting date before comparing models. Results from different setups are not a universal ranking.

Reported results 15

Shown in source observation order, not ranked. Dates retain the precision provided by the source.

Model	Value	Unit	Protocol	Reported date and basis	Source
CenterPoint	18.37	points	Table 5; fixed WorkIng-Highway DAY test split; training=C (W=WorkIng-Highway; D=DrivIng; C=WorkIng-CARLA); LiDAR-only, single sweep, 100m x 100m grid, voxel (0.1,0.1,0.2)m, 25 epochs; nuScenes adapted NDS x100 excludes AAE; W+D+C samples 1/8 CARLA frames per scenario; not night or CARLA-test performance	2026-10-30 · document_publication	WorkIng paper v1 · reported benchmark results
BEVFusion	19.05	points	Table 5; fixed WorkIng-Highway DAY test split; training=C (W=WorkIng-Highway; D=DrivIng; C=WorkIng-CARLA); camera-LiDAR fusion, Swin-Tiny, VoxelNet voxel (0.075,0.075,0.2)m, 50 epochs; nuScenes adapted NDS x100 excludes AAE; W+D+C samples 1/8 CARLA frames per scenario; not night or CARLA-test performance	2026-10-30 · document_publication	WorkIng paper v1 · reported benchmark results
PETR	4.32	points	Table 5; fixed WorkIng-Highway DAY test split; training=C (W=WorkIng-Highway; D=DrivIng; C=WorkIng-CARLA); camera-only, V2-99-eSE, 384x960 input, 75 epochs; nuScenes adapted NDS x100 excludes AAE; W+D+C samples 1/8 CARLA frames per scenario; not night or CARLA-test performance	2026-10-30 · document_publication	WorkIng paper v1 · reported benchmark results
CenterPoint	55.81	points	Table 5; fixed WorkIng-Highway DAY test split; training=D (W=WorkIng-Highway; D=DrivIng; C=WorkIng-CARLA); LiDAR-only, single sweep, 100m x 100m grid, voxel (0.1,0.1,0.2)m, 25 epochs; nuScenes adapted NDS x100 excludes AAE; W+D+C samples 1/8 CARLA frames per scenario; not night or CARLA-test performance	2026-10-30 · document_publication	WorkIng paper v1 · reported benchmark results
BEVFusion	15.7	points	Table 5; fixed WorkIng-Highway DAY test split; training=D (W=WorkIng-Highway; D=DrivIng; C=WorkIng-CARLA); camera-LiDAR fusion, Swin-Tiny, VoxelNet voxel (0.075,0.075,0.2)m, 50 epochs; nuScenes adapted NDS x100 excludes AAE; W+D+C samples 1/8 CARLA frames per scenario; not night or CARLA-test performance	2026-10-30 · document_publication	WorkIng paper v1 · reported benchmark results
PETR	22.37	points	Table 5; fixed WorkIng-Highway DAY test split; training=D (W=WorkIng-Highway; D=DrivIng; C=WorkIng-CARLA); camera-only, V2-99-eSE, 384x960 input, 75 epochs; nuScenes adapted NDS x100 excludes AAE; W+D+C samples 1/8 CARLA frames per scenario; not night or CARLA-test performance	2026-10-30 · document_publication	WorkIng paper v1 · reported benchmark results
CenterPoint	79.4	points	Table 5; fixed WorkIng-Highway DAY test split; training=W (W=WorkIng-Highway; D=DrivIng; C=WorkIng-CARLA); LiDAR-only, single sweep, 100m x 100m grid, voxel (0.1,0.1,0.2)m, 25 epochs; nuScenes adapted NDS x100 excludes AAE; W+D+C samples 1/8 CARLA frames per scenario; not night or CARLA-test performance	2026-10-30 · document_publication	WorkIng paper v1 · reported benchmark results
BEVFusion	76.59	points	Table 5; fixed WorkIng-Highway DAY test split; training=W (W=WorkIng-Highway; D=DrivIng; C=WorkIng-CARLA); camera-LiDAR fusion, Swin-Tiny, VoxelNet voxel (0.075,0.075,0.2)m, 50 epochs; nuScenes adapted NDS x100 excludes AAE; W+D+C samples 1/8 CARLA frames per scenario; not night or CARLA-test performance	2026-10-30 · document_publication	WorkIng paper v1 · reported benchmark results
PETR	43.52	points	Table 5; fixed WorkIng-Highway DAY test split; training=W (W=WorkIng-Highway; D=DrivIng; C=WorkIng-CARLA); camera-only, V2-99-eSE, 384x960 input, 75 epochs; nuScenes adapted NDS x100 excludes AAE; W+D+C samples 1/8 CARLA frames per scenario; not night or CARLA-test performance	2026-10-30 · document_publication	WorkIng paper v1 · reported benchmark results
CenterPoint	79.7	points	Table 5; fixed WorkIng-Highway DAY test split; training=W+D (W=WorkIng-Highway; D=DrivIng; C=WorkIng-CARLA); LiDAR-only, single sweep, 100m x 100m grid, voxel (0.1,0.1,0.2)m, 25 epochs; nuScenes adapted NDS x100 excludes AAE; W+D+C samples 1/8 CARLA frames per scenario; not night or CARLA-test performance	2026-10-30 · document_publication	WorkIng paper v1 · reported benchmark results
BEVFusion	81.33	points	Table 5; fixed WorkIng-Highway DAY test split; training=W+D (W=WorkIng-Highway; D=DrivIng; C=WorkIng-CARLA); camera-LiDAR fusion, Swin-Tiny, VoxelNet voxel (0.075,0.075,0.2)m, 50 epochs; nuScenes adapted NDS x100 excludes AAE; W+D+C samples 1/8 CARLA frames per scenario; not night or CARLA-test performance	2026-10-30 · document_publication	WorkIng paper v1 · reported benchmark results
PETR	47.44	points	Table 5; fixed WorkIng-Highway DAY test split; training=W+D (W=WorkIng-Highway; D=DrivIng; C=WorkIng-CARLA); camera-only, V2-99-eSE, 384x960 input, 75 epochs; nuScenes adapted NDS x100 excludes AAE; W+D+C samples 1/8 CARLA frames per scenario; not night or CARLA-test performance	2026-10-30 · document_publication	WorkIng paper v1 · reported benchmark results
CenterPoint	80.75	points	Table 5; fixed WorkIng-Highway DAY test split; training=W+D+C (W=WorkIng-Highway; D=DrivIng; C=WorkIng-CARLA); LiDAR-only, single sweep, 100m x 100m grid, voxel (0.1,0.1,0.2)m, 25 epochs; nuScenes adapted NDS x100 excludes AAE; W+D+C samples 1/8 CARLA frames per scenario; not night or CARLA-test performance	2026-10-30 · document_publication	WorkIng paper v1 · reported benchmark results
BEVFusion	84.98	points	Table 5; fixed WorkIng-Highway DAY test split; training=W+D+C (W=WorkIng-Highway; D=DrivIng; C=WorkIng-CARLA); camera-LiDAR fusion, Swin-Tiny, VoxelNet voxel (0.075,0.075,0.2)m, 50 epochs; nuScenes adapted NDS x100 excludes AAE; W+D+C samples 1/8 CARLA frames per scenario; not night or CARLA-test performance	2026-10-30 · document_publication	WorkIng paper v1 · reported benchmark results
PETR	52.27	points	Table 5; fixed WorkIng-Highway DAY test split; training=W+D+C (W=WorkIng-Highway; D=DrivIng; C=WorkIng-CARLA); camera-only, V2-99-eSE, 384x960 input, 75 epochs; nuScenes adapted NDS x100 excludes AAE; W+D+C samples 1/8 CARLA frames per scenario; not night or CARLA-test performance	2026-10-30 · document_publication	WorkIng paper v1 · reported benchmark results

What changes in a work zone?

Keep scenes and splits separate

The primary score has one scope

Read the training data labels

Publication versus runnable resources

Table 4 · within-domain detection

Table 5 · Day test by training source

Table 5 · Night test by training source

Table 6 · separate ROADWork transfer test

Table 7 · logged real-world planning

Table 8 · reactive CARLA planning

Table 9 · planning transfer to real data

Table 12 · per-class detection

Original publication

Evidence available for this benchmark

Start with these resources

Evaluation context and evidence

Reported results 15

Explore related benchmarks

Put this benchmark in context

Five new benchmark pages: agent safety, cloud operations and video

Six new benchmark pages: context, agents and executable tasks

Six Navers Lab / Einsia benchmarks and the work they test

AI benchmarks and leaderboards are different things

How to compare AI benchmark scores without mixing protocols

Four new agent benchmarks: adaptation, safety and video

Continue exploring