Robustness Emerges Early in Training Dynamics, but Is Not Preserved

Robustness to natural corruptions remains a fundamentalchallenge for deep neural networks. In this paper, we identify a robustnessfading phenomenon where shallow layers spontaneously develop robustrepresentations and flat loss landscapes in early training, yet these prop-erties are not preserved during standard convergence. To address this,we propose a framework that performs strategic interventions on train-ing dynamics to stabilize the empirically identified early-emergent robustpriors. Our approach includes two parameter-free strategies: Early-PhaseStabilization (EPS) and Asymmetric Weight Reversion (AWR), whichstabilize or recover robust shallow configurations without modifying themodel architecture or introducing learnable parameters. Extensive exper-iments demonstrate the efficacy of our framework across various bench-marks and architectures, yielding significant gains in downstream trans-fer, dynamic adaptation, and diverse computer vision applications.