LaGen: Towards Autoregressive LiDAR Scene Generation

Generative world models for autonomous driving (AD) are ofgreat value in applications such as data augmentation, closed-loop sim-ulation, and safety-critical scenario evaluation. Unlike the widely stud-ied image modality, in this work we explore generative world modelsfor LiDAR data. Existing generation methods for LiDAR predominantlyfocus on single frame generation or lack the capacity for interactive sim-ulation, while existing prediction approaches require multiple frames ofhistorical input and can only deterministically predict multiple framesat once. Both paradigms fail to support long-horizon interactive gen-eration. To this end, we introduce LaGen, which, to the best of ourknowledge is the first autoregressive framework capable of generatinglong-horizon LiDAR scenes in a frame-by-frame, interactive manner. La-Gen is able to take a single-frame input as a starting point and effectivelyutilize bounding box information as conditions to generate high-fidelity4D scene. In addition, we introduce a scene decoupling estimation moduleto enhance the model’s interactive generation capability for object-levelcontent, as well as a noise modulation module to mitigate error accumu-lation during long-horizon generation. We extensively evaluate LaGen’sperformance in controlled data generation and long-horizon scene genera-tion on the nuScenes dataset. The experimental results demonstrate thatLaGen achieves state-of-the-art performance, especially on later frames.The code is publicly available at: https://github.com/szzhou88/LaGen.