Model reports

PanoVideoBench

Follow people and objects through a 360-degree video, including evidence that moves between viewing directions and moments. A model that recognizes one frame may still lose the story.

Original paper Official project Official dataset Catalog source

AI Benchmark Radar / Equality Machine · Reviewed 2026-11-03

Original paper v1, submitted 30 October 2026; reviewed and added 3 November. Full-benchmark accuracy and labeled input/training ablations.

What does a question require?

Choose one of four answers using evidence across the viewing sphere and timeline. The 1,000 human-annotated questions use 223 videos and cover appearance, space, action, motion and events.

PanoVideoBench original paper v1 · PanoVideoBench · author annotation dataset

Hold the viewing budget fixed

The default uses the original equirectangular video: uniform sampling at 1 fps, at most 128 frames, 1,280-pixel long edge, model-default temperature and up to 32,768 output tokens including thinking.

PanoVideoBench · evaluation and input protocol

Overall and category scores differ

Overall weights every question equally. Category sizes differ; their simple mean is not Overall. The primary results include all distinct overall input/training variants; repeated baseline rows are recorded once.

PanoVideoBench · Table 3 · PanoVideoBench · Table 4 · PanoVideoBench · Table 5 · PanoVideoBench · author annotation dataset

Where to obtain the benchmark

The author dataset publishes annotations and video identifiers; obtain source videos separately. Gold answers and diagnostic labels are evaluation annotations, not model inputs.

PanoVideoBench · author annotation dataset

Table 3 · full comparison

Eleven baselines and the authors’ trained system share the default video input.

Table 3 · full comparison
Model	Overall (%)	APP (%)	SPA (%)	ACT (%)	MOT (%)	TEM (%)
Gemini 3.7 Flash	53.1	65.7	49.5	52.4	40.9	36.5
GPT-5.6-sol	49.6	59.9	46.0	51.9	33.5	41.9
Gemma-4-31B-it	47.0	58.4	48.5	44.6	34.8	27.0
Nemotron Omni 30B-A3B	46.3	59.9	46.0	41.6	31.1	35.1
InternVL3.5-8B	44.9	53.8	36.1	49.4	34.1	39.2
Qwen3.8-27B	44.8	54.7	44.1	46.8	31.1	27.0
Qwen3.5-9B	43.2	55.4	36.6	42.5	35.4	27.0
Eagle2.5-8B	39.6	50.5	35.6	38.6	32.3	21.6
MiniCPM-V-4.6	39.0	48.9	37.1	34.3	34.1	25.7
GLM-4.1V-9B-Thinking	38.0	46.5	38.6	32.2	31.1	32.4
PanoWorld-Hstar	42.9	51.7	45.0	36.5	34.1	37.8
PanoVideoReasoner	55.7	59.6	52.5	60.1	48.8	48.6

Only Overall enters the primary series. Category results are subsets.

PanoVideoBench · Table 3

Table 4a · ERP resolution

All cells are printed percentages. Qwen is Qwen3.8-27B and Gemma is Gemma-4-31B-it; training variants start from Qwen3.5-9B.

Table 4a · ERP resolution
Metric	Qwen 640	Qwen 1280	Qwen 1920	Gemma 640	Gemma 1280	Gemma 1920
Overall	42.3	44.8	46.2	46.0	47.0	47.2
APP	46.8	54.7	56.0	57.2	58.4	58.1
SPA	44.6	44.1	48.5	47.0	48.5	48.0
ACT	45.9	46.8	48.5	42.9	44.6	45.1
MOT	31.1	31.1	32.9	34.1	34.8	32.9
TEM	29.7	27.0	18.9	29.7	27.0	35.1

ERP is the default projection; 8V is an eight-view mosaic; Ovl adds orientation markers. Unif./Q-g./EP specify uniform, question-guided or evidence-privileged teacher views.

PanoVideoBench · Table 4

Table 4b · input representation

All cells are printed percentages. Qwen is Qwen3.8-27B and Gemma is Gemma-4-31B-it; training variants start from Qwen3.5-9B.

Table 4b · input representation
Metric	Qwen ERP	Qwen 8V	Qwen Ovl	Gemma ERP	Gemma 8V	Gemma Ovl
Overall	44.8	41.0	44.5	47.0	40.9	44.5
APP	54.7	47.7	48.6	58.4	43.7	45.6
SPA	44.1	44.6	48.0	48.5	40.6	50.5
ACT	46.8	42.5	45.1	44.6	42.1	43.3
MOT	31.1	26.8	34.1	34.8	37.8	42.1
TEM	27.0	28.4	37.8	27.0	32.4	32.4

ERP is the default projection; 8V is an eight-view mosaic; Ovl adds orientation markers. Unif./Q-g./EP specify uniform, question-guided or evidence-privileged teacher views.

PanoVideoBench · Table 4

Table 5a · supervision

All cells are printed percentages. Qwen is Qwen3.8-27B and Gemma is Gemma-4-31B-it; training variants start from Qwen3.5-9B.

Table 5a · supervision
Metric	Base	Answer	CoT
Overall	43.2	45.6	51.8
APP	55.4	55.0	57.2
SPA	36.6	41.1	45.0
ACT	42.5	45.5	57.1
MOT	35.4	36.0	45.1
TEM	27.0	37.8	44.6

ERP is the default projection; 8V is an eight-view mosaic; Ovl adds orientation markers. Unif./Q-g./EP specify uniform, question-guided or evidence-privileged teacher views.

PanoVideoBench · Table 5

Table 5b · teacher views

All cells are printed percentages. Qwen is Qwen3.8-27B and Gemma is Gemma-4-31B-it; training variants start from Qwen3.5-9B.

Table 5b · teacher views
Metric	CoT	Unif.	Q-g.	EP
Overall	51.8	50.9	53.9	55.7
APP	57.2	57.8	58.4	59.6
SPA	45.0	46.5	49.0	52.5
ACT	57.1	55.4	58.8	60.1
MOT	45.1	41.5	47.0	48.8
TEM	44.6	39.2	47.3	48.6

ERP is the default projection; 8V is an eight-view mosaic; Ovl adds orientation markers. Unif./Q-g./EP specify uniform, question-guided or evidence-privileged teacher views.

PanoVideoBench · Table 5

Original publication

Original paper v1, submitted 30 October 2026; reviewed and added 3 November. Full-benchmark accuracy and labeled input/training ablations.

Reviewed 2026-11-03

Evidence available for this benchmark

This source record contains 24 numeric observations. The source distinguishes 16 model identities or configurations. A model may appear more than once; check the version, tools and run budget before comparing. Read results and row-level evidence →

Reported metric: Overall choice accuracy (all 1,000 questions)
Recorded unit: percent
Metric direction: higher is better

Start with these resources

Catalog source · arxiv.org

Cited documents and dates (1)

PanoVideoBench paper v1 · reported benchmark results · 2026-10-30

Evaluation context and evidence

Source: Model reports
Publisher: Not recorded
Benchmark release: 2026-10-30
Collected: Not recorded
Modality: Not recorded

These are source-reported observations, not independent reruns by this site. Check the task version, evaluation protocol, unit and reporting date before comparing models. Results from different setups are not a universal ranking.

Reported results 24

Shown in source observation order, not ranked. Dates retain the precision provided by the source.

Model	Value	Unit	Protocol	Reported date and basis	Source
Gemma-4-31B-it	44.5	percent	Full 1,000-question choice accuracy; ERP with fixed question-independent orientation overlay; uniform whole-video 1 fps, at most 128 frames; long edge 1280px; model default temperature; max 32768 output tokens incl thinking; Table 4b Ovl input; see Appendix C.3	2026-10-30 · document_publication	PanoVideoBench paper v1 · reported benchmark results
Qwen3.8-27B	44.5	percent	Full 1,000-question choice accuracy; ERP with fixed question-independent orientation overlay; uniform whole-video 1 fps, at most 128 frames; long edge 1280px; model default temperature; max 32768 output tokens incl thinking; Table 4b Ovl input; see Appendix C.3	2026-10-30 · document_publication	PanoVideoBench paper v1 · reported benchmark results
Gemma-4-31B-it	40.9	percent	Full 1,000-question choice accuracy; eight-view compass perspective mosaic; uniform whole-video 1 fps, at most 128 frames; long edge 1280px; model default temperature; max 32768 output tokens incl thinking; Table 4b 8V input; see Appendix C.3	2026-10-30 · document_publication	PanoVideoBench paper v1 · reported benchmark results
Qwen3.8-27B	41.0	percent	Full 1,000-question choice accuracy; eight-view compass perspective mosaic; uniform whole-video 1 fps, at most 128 frames; long edge 1280px; model default temperature; max 32768 output tokens incl thinking; Table 4b 8V input; see Appendix C.3	2026-10-30 · document_publication	PanoVideoBench paper v1 · reported benchmark results
Eagle2.5-8B	39.6	percent	Full 1,000-question choice accuracy; original ERP video; uniform whole-video 1 fps, at most 128 frames; long edge 1280px; model default temperature; max 32768 output tokens incl thinking; Table 3 main comparison	2026-10-30 · document_publication	PanoVideoBench paper v1 · reported benchmark results
GLM-4.1V-9B-Thinking	38.0	percent	Full 1,000-question choice accuracy; original ERP video; uniform whole-video 1 fps, at most 128 frames; long edge 1280px; model default temperature; max 32768 output tokens incl thinking; Table 3 main comparison	2026-10-30 · document_publication	PanoVideoBench paper v1 · reported benchmark results
GPT-5.6-sol	49.6	percent	Full 1,000-question choice accuracy; original ERP video; uniform whole-video 1 fps, at most 128 frames; long edge 1280px; model default temperature; max 32768 output tokens incl thinking; Table 3 main comparison	2026-10-30 · document_publication	PanoVideoBench paper v1 · reported benchmark results
Gemini 3.7 Flash	53.1	percent	Full 1,000-question choice accuracy; original ERP video; uniform whole-video 1 fps, at most 128 frames; long edge 1280px; model default temperature; max 32768 output tokens incl thinking; Table 3 main comparison	2026-10-30 · document_publication	PanoVideoBench paper v1 · reported benchmark results
Gemma-4-31B-it	47.0	percent	Full 1,000-question choice accuracy; original ERP video; uniform whole-video 1 fps, at most 128 frames; long edge 1280px; model default temperature; max 32768 output tokens incl thinking; Table 3 main comparison	2026-10-30 · document_publication	PanoVideoBench paper v1 · reported benchmark results
InternVL3.5-8B	44.9	percent	Full 1,000-question choice accuracy; original ERP video; uniform whole-video 1 fps, at most 128 frames; long edge 1280px; model default temperature; max 32768 output tokens incl thinking; Table 3 main comparison	2026-10-30 · document_publication	PanoVideoBench paper v1 · reported benchmark results
MiniCPM-V-4.6	39.0	percent	Full 1,000-question choice accuracy; original ERP video; uniform whole-video 1 fps, at most 128 frames; long edge 1280px; model default temperature; max 32768 output tokens incl thinking; Table 3 main comparison	2026-10-30 · document_publication	PanoVideoBench paper v1 · reported benchmark results
Nemotron Omni 30B-A3B	46.3	percent	Full 1,000-question choice accuracy; original ERP video; uniform whole-video 1 fps, at most 128 frames; long edge 1280px; model default temperature; max 32768 output tokens incl thinking; Table 3 main comparison	2026-10-30 · document_publication	PanoVideoBench paper v1 · reported benchmark results
PanoVideoReasoner	55.7	percent	Full 1,000-question choice accuracy; original ERP video; uniform whole-video 1 fps, at most 128 frames; long edge 1280px; model default temperature; max 32768 output tokens incl thinking; Table 3 main comparison	2026-10-30 · document_publication	PanoVideoBench paper v1 · reported benchmark results
PanoWorld-Hstar	42.9	percent	Full 1,000-question choice accuracy; original ERP video; uniform whole-video 1 fps, at most 128 frames; long edge 1280px; model default temperature; max 32768 output tokens incl thinking; Table 3 main comparison	2026-10-30 · document_publication	PanoVideoBench paper v1 · reported benchmark results
Qwen3.5-9B	43.2	percent	Full 1,000-question choice accuracy; original ERP video; uniform whole-video 1 fps, at most 128 frames; long edge 1280px; model default temperature; max 32768 output tokens incl thinking; Table 3 main comparison	2026-10-30 · document_publication	PanoVideoBench paper v1 · reported benchmark results
Qwen3.8-27B	44.8	percent	Full 1,000-question choice accuracy; original ERP video; uniform whole-video 1 fps, at most 128 frames; long edge 1280px; model default temperature; max 32768 output tokens incl thinking; Table 3 main comparison	2026-10-30 · document_publication	PanoVideoBench paper v1 · reported benchmark results
Qwen3.5-9B · CoT SFT	51.8	percent	Full 1,000-question choice accuracy; original ERP video; uniform whole-video 1 fps, at most 128 frames; long edge 1280px; model default temperature; max 32768 output tokens incl thinking; Table 5a CoT SFT; trained by paper authors on PanoVideoCoT	2026-10-30 · document_publication	PanoVideoBench paper v1 · reported benchmark results
Qwen3.5-9B · answer-only SFT	45.6	percent	Full 1,000-question choice accuracy; original ERP video; uniform whole-video 1 fps, at most 128 frames; long edge 1280px; model default temperature; max 32768 output tokens incl thinking; Table 5a answer-only SFT; trained by paper authors on PanoVideoCoT	2026-10-30 · document_publication	PanoVideoBench paper v1 · reported benchmark results
Qwen3.5-9B · question-guided OPSD	53.9	percent	Full 1,000-question choice accuracy; original ERP video; uniform whole-video 1 fps, at most 128 frames; long edge 1280px; model default temperature; max 32768 output tokens incl thinking; Table 5b CoT SFT plus question-guided teacher views; trained by paper authors on PanoVideoCoT	2026-10-30 · document_publication	PanoVideoBench paper v1 · reported benchmark results
Qwen3.5-9B · uniform-view OPSD	50.9	percent	Full 1,000-question choice accuracy; original ERP video; uniform whole-video 1 fps, at most 128 frames; long edge 1280px; model default temperature; max 32768 output tokens incl thinking; Table 5b CoT SFT plus uniform teacher views; trained by paper authors on PanoVideoCoT	2026-10-30 · document_publication	PanoVideoBench paper v1 · reported benchmark results
Gemma-4-31B-it	47.2	percent	Full 1,000-question choice accuracy; original ERP video; uniform whole-video 1 fps, at most 128 frames; long edge 1920px; model default temperature; max 32768 output tokens incl thinking; Table 4a resolution variant	2026-10-30 · document_publication	PanoVideoBench paper v1 · reported benchmark results
Qwen3.8-27B	46.2	percent	Full 1,000-question choice accuracy; original ERP video; uniform whole-video 1 fps, at most 128 frames; long edge 1920px; model default temperature; max 32768 output tokens incl thinking; Table 4a resolution variant	2026-10-30 · document_publication	PanoVideoBench paper v1 · reported benchmark results
Gemma-4-31B-it	46.0	percent	Full 1,000-question choice accuracy; original ERP video; uniform whole-video 1 fps, at most 128 frames; long edge 640px; model default temperature; max 32768 output tokens incl thinking; Table 4a resolution variant	2026-10-30 · document_publication	PanoVideoBench paper v1 · reported benchmark results
Qwen3.8-27B	42.3	percent	Full 1,000-question choice accuracy; original ERP video; uniform whole-video 1 fps, at most 128 frames; long edge 640px; model default temperature; max 32768 output tokens incl thinking; Table 4a resolution variant	2026-10-30 · document_publication	PanoVideoBench paper v1 · reported benchmark results

What does a question require?

Hold the viewing budget fixed

Overall and category scores differ

Where to obtain the benchmark

Table 3 · full comparison

Table 4a · ERP resolution

Table 4b · input representation

Table 5a · supervision

Table 5b · teacher views

Original publication

Evidence available for this benchmark

Start with these resources

Evaluation context and evidence

Reported results 24

Explore related benchmarks

Put this benchmark in context

Five new benchmark pages: agent safety, cloud operations and video

Six new benchmark pages: context, agents and executable tasks

Six Navers Lab / Einsia benchmarks and the work they test

AI benchmarks and leaderboards are different things

How to compare AI benchmark scores without mixing protocols

Four new agent benchmarks: adaptation, safety and video

Continue exploring