Spectral Prior for Reducing Exposure Bias in Diffusion Models

Diffusion models typically suffer from error accumulationduring iterative sampling, commonly referred to as exposure bias. Wereveal systematic frequency-dependent discrepancies between trainingand inference, which can be interpreted as frequency-dependent SNRerror. Crucially, the direction of this mismatch varies across modelsand timesteps, indicating that fixed correction rules do not generalize.We propose Spectral Alignment (SPA), a lightweight, guidance-basedmethod that calibrates the power spectrum of intermediate predictionsto a pre-computed prior. Our approach consists of two stages: (1) of-fline fitting of a parametric spectrum model from training data, and (2)inference-time guidance via efficient FFT-based gradient computation.SPA introduces minimal computational overhead (3-4%) and is com-plementary to Classifier-Free Guidance (CFG). We demonstrate consis-tent improvements across diverse architectures, from pixel-space mod-els (DDPM, ADM) to latent diffusion models (SD2.0, SDXL) and flow-matching models (SD3.5, FLUX).