Rethinking Token Reduction for Diffusion Models via Output-Similarity-Awareness

Diffusion Transformers (DiTs) achieve superior image gener-ation quality but suffer from quadratic computational complexity rela-tive to token count. While various token reduction (TR) methods havebeen proposed to mitigate this cost, they overlook the primary objectiveof generative models: minimizing recovery error, which requires reflect-ing output token similarity. Specifically, they rely solely on input tokensimilarity inherited from reduction-only ViT paradigms, leading to a fun-damental misalignment with this objective.To bridge this gap, we propose DiTo, a novel TR paradigm that shiftsthe focus toward output-centric token reduction. Based on the observa-tion that output token similarity is consistently preserved across adjacenttimesteps, DiTo utilizes prior-step similarities as an effective proxy toestablish token correspondences at a Matching timestep, and reuses theresulting correspondences across multiple subsequent Reduction steps.To optimize this interleaved scheduling, we propose Pair Match Ra-tio (PMR)-guided Interval Scheduling to determine the optimal match-ing frequency. Furthermore, to mitigate localized approximation errorsand resulting blocking artifacts caused by repeated reuse, we proposeFrequency-aware Token Matching by incorporating a selection-frequencypenalty. Extensive experiments demonstrate that DiTo consistently out-performs existing TR methods with 1.6–3.9 dB higher PSNR at compa-rable speedups, achieving a superior Pareto frontier.