Layer-Aware Video Composition via Split-then-Merge

We present Split-then-Merge (StM), a controllable genera-tive video composition framework that minimizes reliance on annotateddatasets and handcrafted rules. Instead of requiring manual supervision,StM decomposes unlabeled videos into dynamic foreground and back-ground layers. By self-composing these elements, the model learns tosynthesize complex interactions between moving subjects and diversescenes, capturing the intricate dynamics necessary for high-x001Cdelity videogeneration. Specix001Ccally, StM introduces a transformation-aware train-ing pipeline utilizing multi-layer fusion and augmentation to address af-fordance challenges in video composition. An identity-preservation lossfurther maintains foreground x001Cdelity during blending. Extensive exper-iments show that StM outperforms state-of-the-art methods on bothquantitative benchmarks and qualitative evaluations, including humanstudies and Vision-Language Model assessments. Finally, we release StM-50K, the x001Crst multi-layer video dataset, to facilitate future research ingenerative video composition. Data, code, and more details are availableat our project page.Fig. 2: Video Composition. Given input foreground and background videos, image-based methods (a)x0015(b) use only the x001Crst frame, while (c)x0015(e) take full video inputs.(a) Object insertion [59] followed by Image-to-Video (I2V) and (b) end-to-end I2Vcomposition SkyReels [13] fails to retain motion due to lack of video access. (c) Manualcopy-paste preserves motion but violates ax001Bordance (swan placed on ground). (d) Naivegenerative composition yields appearance and motion drift (e.g ., black swan turnswhite). (e) Our method preserves identity and motion, and achieves ax001Bordance-awareplacement with realistic blending (swan placed in water with wave and shadows).