SIMSplat: Language-Aligned 4D Gaussian Splatting for Driving Scenario Generation

Driving scene manipulation using real-world sensor data hasemerged as a promising alternative to traditional driving simulators. De-spite advances in language control and neural scene representations, ex-isting methods treat grounding, editing, and simulation as loosely con-nected stages, relying on heuristic object localization, manual guidance,and single-agent validation—thereby constraining semantic expressive-ness and hindering scalable, reactive scenario generation. We introduceSIMSplat, a driving scene editor built on scene-graph-based 4D Gaus-sian Splatting augmented with language-aligned features. By embed-ding appearance, motion, and location semantics directly into Gaus-sian scene-graph nodes, SIMSplat makes reconstructed scenes queryablethrough free-form natural language, bridging language understanding toobject-level editing and multi-agent simulation within a single frame-work. Building on this language-grounded scene graph, SIMSplat sup-ports diverse edits including fine-grained pedestrian manipulation, whilea multi-agent path refinement module propagates changes across allagents to ensure reactive, physically plausible simulations. The pipelinefurther integrates with Vision-Language Models for automated scenariomining. Experiments show that SIMSplat more than doubles baselinegrounding accuracy, achieves the highest task completion rate, and pro-duces the lowest failure rates across diverse driving scenarios.