Adaptive Noise Covariance Scheduling under Riemannian Metrics for Diffusion Models

Diffusion models typically reconstruct coarse, low-frequency structure before recovering high-frequency detail during the reverse denoising. However, the standard forward process injects Gaussian white noise that corrupts all frequencies uniformly, creating a mismatch between forward corruption and reverse denoising and hindering the recovery of fine detail. We address this mismatch with an adaptive noise covariance schedule that evolves along the geodesic on the Symmetric Positive Definite (SPD) manifold under the Bures-Wasserstein (BW) metric. Specifically, we evolve the covariance along the BW geodesic from a blue-noise endpoint to a white-noise endpoint, so the noise power spectrum transitions smoothly from high-frequency emphasis to uniform. This transition better aligns forward corruption with reverse denoising and enhances textural detail. We provide theoretical and quantitative analyses showing that this transition path yields smoother noise power spectrum evolution and lower noise power fluctuations. In addition, we adapt the transition speed to the characteristics of each image, yielding finer forward-reverse alignment and improved perceptual quality. Experiments on multiple datasets show consistent gains over deterministic diffusion baselines in FID and KID. Our code and pretrained models are available at https://github.com/BolinDeng/RMDM.