Out of Sight, Out of Mind? Evaluating State Evolution in Video World Models

Evolutions in the world, such as water pouring or ice melt-ing, happen regardless of being observed. Video world models generate“worlds” via 2D frame observations. Can these generated “worlds” evolveregardless of observation? To probe this question, we design a benchmarkto evaluate whether video world models can decouple state evolutionfrom observation. Our benchmark, StEvo-Bench, applies observationcontrol to evolving processes via instructions of occluder insertion, turn-ing off the light, or specifying camera “lookaway” trajectories. By evalu-ating video models with and without camera control for a diverse set ofnaturally-occurring evolutions, we expose their limitations in decouplingstate evolution from observation. StEvo-Bench proposes an evaluationprotocol to automatically detect and disentangle failure modes of videoworld models across key aspects of natural state evolution. Analysis ofStEvo-Bench results provide new insight into potential data and ar-chitecture bias of present-day video world models. Benchmark website:https://glab-caltech.github.io/STEVOBench/.