OmniForcing: Unleashing Real-time Joint Audio-Visual Generation

Recent joint audio-visual diffusion models achieve remark-able generation quality but suffer from high latency due to their bidirec-tional attention dependencies, hindering real-time applications. We pro-pose OmniForcing, the first framework to distill an offline, dual-streambidirectional diffusion model into a high-fidelity streaming autoregressivegenerator. However, naively applying causal distillation to such dual-stream architectures triggers severe training instability, due to the ex-treme temporal asymmetry between modalities and the resulting tokensparsity. We address the inherent information density gap by introducingan Asymmetric Block-Causal Alignment with a zero-truncation GlobalPrefix that prevents multi-modal synchronization drift. The gradient ex-plosion caused by extreme audio token sparsity during the causal shift isfurther resolved through an Audio Sink Token mechanism equipped withan Identity RoPE constraint. Finally, a Joint Self-Forcing Distillationparadigm enables the model to dynamically self-correct cumulative cross-modal errors from exposure bias during long rollouts. Empowered by amodality-independent rolling KV-cache inference scheme, OmniForcingachieves state-of-the-art streaming generation at ∼25 FPS on a singleGPU, maintaining multi-modal synchronization and visual quality on parwith the bidirectional teacher. Project Page: https://omniforcing.com.