Predictive Structure Improves Video Diffusion Dynamics

Video diffusion models have achieved remarkable visual fi-delity, yet often struggle to adhere to basic physical constraints, suchas support, motion consistency, and object permanence, leading to im-plausible dynamics even in short clips. We revisit this challenge throughthe lens of the Information Bottleneck principle, arguing that physicallygrounded video generation should prioritize time-predictive representa-tions over high-entropy appearance variations. Building on this perspec-tive, we propose Latent Dynamics Optimization (LDO), a post-trainingframework that leverages a predictive latent world model (V-JEPA 2)to guide a pretrained video diffusion model towards dynamics-consistentgeneration without altering its sampling process. LDO operates on twolevels: (1) aligning Gram-matrix geometry of intermediate diffusion fea-tures with the latent world model’s casually predicted dynamics, and(2) encouraging temporally consistent rollouts with Group-Relative Pol-icy Optimization (GRPO) using the world model as a dynamics critic.Experiments on VideoPhy and PisaBench show that LDO substantiallyimproves physical commonsense, object permanence, and trajectory fi-delity while preserving visual quality, suggesting that predictive latentsupervision offers a practical route to make video generators not onlyphotorealistic but also physically legible.