LUA: Latent Upscaling Adapter for Diffusion-Based Image Synthesis

Generating high-resolution images with latent diffusion models is limited by the cost of high-resolution denoising, while post-hoc super-resolution introduces artifacts and additional latency by operating after decoding. We present the Latent Upscaling Adapter (LUA), a lightweight module that performs super-resolution directly on the generator’s latent code before the final VAE decoding step. LUA integrates as a drop-in component requiring no modifications to the base model or additional diffusion stages, enabling high-resolution synthesis through a single feed-forward pass in latent space. A shared Swin-style backbone with scale-specific pixel-shuffle heads supports ×2 and ×4 factors with nearly 3× lower decoding and upscaling time. Because LUA operates directly on latents, it transfers across diffusion systems–including SDXL, SD3, and FLUX–through architecture-level reuse, adapting only the input convolution with brief fine-tuning rather than retraining from scratch. Extensive experiments demonstrate that LUA closely matches the fidelity of native high-resolution generation at 2K and 4K while offering a practical and efficient path to scalable image synthesis.