Schroedinger’s Cat: Probabilistic Representation and Prediction of Potential Scene Kinematics

Predicting how a scene may evolve from partial observationsrequires reasoning about multiple possible futures rather than committingto a single trajectory. Existing approaches either generate appearance-dominated video predictions or sample a small number of trajectorieswithout explicitly modeling the distribution of possible motion. We intro-duce Goal Aware Representations of Future kInEmatic Latent Distribu-tions (GARFIELD), a probabilistic model of scene kinematics that learnsa structured spatio-temporal latent representation of the distribution overpossible futures given an image and optional spatio-temporally sparseconstraints. The same latent representation enables both joint samplingof all trajectories and direct access to the underlying motion distributionthrough an efficient deterministic density decoder. As a result, uncer-tainty about future motion can be localized to specific scene elementsand timesteps and progressively refined through additional constraints.Experiments demonstrate strong motion planning performance competi-tive with large video generation models while sampling trajectories 97×faster. Our method further estimates motion densities two orders of mag-nitude faster than Monte-Carlo sampling from motion generation models,enabling interactive exploration and uncertainty-aware planning.Project page: https://compvis.github.io/schroedingers_cat/