From Synchrony to Sequence: Exo-to-Ego Generation via Interpolation

Exo-to-Ego video generation aims to synthesize a first-personvideo from a synchronized third-person view and corresponding cam-era poses. While paired supervision is available, synchronized exo-egodata inherently introduces substantial spatio-temporal and geometricdiscontinuities, violating the smooth-motion assumptions of standardvideo generation benchmarks. We identify this synchronization-inducedjump as the central challenge and propose Syn2Seq-Forcing, a sequen-tial formulation that interpolates between the source and target videosto form a single continuous signal. By reframing Exo2Ego as sequentialsignal modeling rather than a conventional condition–output task, ourapproach enables diffusion-based sequence models, e.g. Diffusion Forc-ing Transformers (DFoT), to capture coherent transitions across framesmore effectively. Empirically, we show that interpolating only the videos,without performing pose interpolation already produces significant im-provements, emphasizing that the dominant difficulty arises from spatio-temporal discontinuities. Beyond immediate performance gains, this for-mulation establishes a general and flexible framework capable of uni-fying both Exo2Ego and Ego2Exo generation within a single contin-uous sequence model, providing a principled foundation for future re-search in cross-view video synthesis. The code will be released at https://github.com/insait-institute/Syn2Seq.