Test-Time Registers as Global Priors for Tokenized Image Generation

Attention-based models often develop attention sinks, wherea small number of tokens repeatedly attract attention and accumulateunusually large activations. In vision transformers, these outliers areclosely related to registers, which have been diagnostically linked toglobal, low-frequency image structure. Existing work has largely studiedregisters through interpretability analyses and linear probes, leaving openwhether they can be operationalized as plug-and-play signals for genera-tion without retraining. We revisit this question in tokenized image gen-eration. Using OpenCLIP and DINOv2 on ImageNet, we find that test-time register features exhibit stronger low-frequency concentration thanboth [CLS] readouts and patch-mean features, and show a consistent (al-beit moderate) correlation with pixel-space DCT low-frequency energy.Motivated by these diagnostics, we introduce RegToken, a training-free procedure that converts register structure into a small set of globalprior tokens by (i) NFN-based layer localization, (ii) TokenRank-guidedsubspace extraction, and (iii) a projection-and-conservation update onthe register subspace. Inserted into a frozen compact 1D token gener-ation pipeline, RegToken improves ImageNet generation and alignmentmetrics (e.g., FID-5k 20.5 → 20.1, SigLIP 3.6 → 3.9) without modify-ing pretrained weights, and accelerates test-time optimization (Steps@τ74 → 52). Overall, our results suggest that structures often viewed asattention artifacts can be repurposed as lightweight global priors for to-kenized generation.