The paper v1 evaluation of eight image-to-video models on 40 controlled physical tasks. The published overall measure is a composite score on a 0–100 scale.
Controlled physical experiments
The 40 tasks cover nine physical families, including mechanics, optics, buoyancy, phase change, electromagnetism and fluids. Examples compare pendulum periods, communicating-vessel levels and capillary rise. Models receive the same initial image and prompt for each task; evaluation does not require a reference video.
World Models’ Last Exam in Physics paper v1 · World Models’ Last Exam in Physics official project
Measure the physical evidence
The evaluator separates consistency and observability C from independently measured physical correctness P. Tracking and image-analysis tools extract quantities such as trajectories, angles, periods or event timing. Fixed task-specific rules map deviations from expected relationships into physical scores.
Read the composite score correctly
Each video uses S = 0.15C + 0.85P × I(C ≥ 80), then the overall score averages videos. A failed consistency gate removes the physical contribution, not necessarily the entire score. The table checkmark means all available samples in a model–task pair satisfy C ≥ 80; it is not a task-success label.
World Models’ Last Exam in Physics paper v1 · Official paper results JSON
Four samples, model-specific video settings
The published table covers 1,280 videos: eight models × 40 tasks × four samples. Seeds 42–45 are fixed where supported, but the Seedance API adapter uses them only as sample labels. Resolutions and frame counts vary by model, so these generation settings belong with the reported score.
World Models’ Last Exam in Physics paper v1 · Generation protocol and model settings
Visibility and measurement remain limits
Occlusion, unclear geometry or incomplete events can make physical evidence unreliable. Synthetic-video checks validate measurement under controlled conditions, not every complex scene. The 15 easy, 15 medium and 10 hard groups are empirical labels derived from the evaluated models, and do not constitute an independent difficulty standard.