ICDepth: Taming Video Diffusion Models for Video Depth Estimation via In-Context Conditioning

Monocular video depth estimation requires temporal con-sistency, geometric accuracy, and generalization across diverse scenar-ios—yet existing methods struggle to achieve all three simultaneously.Discriminative models excel at per-frame accuracy but suffer from tem-poral drift due to limited context windows, while generative methodsimprove consistency and generalization at the cost of extensive train-ing data (10M+ samples) and lack of geometric precision. In responseto these issues, we introduce ICDepth, a framework that adapts pre-trained text-to-video diffusion transformers for video depth estimationvia In-Context Conditioning (ICC), leveraging their rich spatial-temporalpriors. To address key challenges in transferring ICC from generationto dense prediction, we propose: (1) SAND-Attention, which ensuresprecise spatial-temporal alignment via shared RoPE and enforces uni-directional attention to prevent noise contamination; (2) SRFM, whichinjects DINOv2 semantic and resolution priors to enhance geometric pre-cision. ICDepth achieves state-of-the-art results on multiple benchmarkswith remarkable data efficiency, trained on only 0.8M frames (6–13×less than competing generative methods), while demonstrating strongzero-shot generalization to diverse domains.