Autoregressive Image Generation Needs Only a Few Lines of Cached Tokens

Autoregressive (AR) visual generation has emerged as a powerful paradigm for image and multimodal synthesis, owing to its scalability and generality. However, existing AR image generation suffers from severe memory bottlenecks due to the need to cache all previously generated visual tokens during decoding, leading to both high storage requirements and low throughput. In this paper, we introduce LineAR, a novel, training-free progressive key-value (KV) cache compression pipeline for AR image generation. By exploiting the intrinsic characteristics of visual attention, LineAR manages the cache from a 2D line-level perspective, preserving the visual dependency regions while progressively evicting less-informative tokens under inter-line attention guidance, leveraging the spatial continuity prior of images to ensure that each eviction step is harmless to subsequent line generation. Experiments across seven AR image generation models validate that LineAR achieves lossless or even improved generation quality with only a few cached lines. It improves ImageNet FID from 2.77 to 2.68 on LlamaGen-XL and COCO FID from 23.85 to 22.86 on Janus-Pro-1B at 1/6 budget ratio, and also improves DPG and HPSv2.1 scores on Lumina-mGPT-768 with just 1/8 KV cache. Additionally, LineAR achieves significant memory and throughput gains across devices, e.g., up to 67.06% memory reduction and 4.17× speedup on LlamaGen-XL, and 73.65% memory reduction and 4.34× speedup on Janus-Pro-1B, evaluated on NVIDIA RTX PRO 6000 GPUs.