ExploreVLA: Dense World Modeling and Exploration for End-to-End Autonomous Driving

End-to-end autonomous driving models based on Vision-Language-Action (VLA) architectures have shown promising results bylearning driving policies through behavior cloning on expert demonstra-tions. However, imitation learning inherently limits the model to repli-cating observed behaviors without exploring diverse driving strategies,leaving it brittle in novel or out-of-distribution scenarios. Reinforcementlearning (RL) offers a natural remedy by enabling policy explorationbeyond the expert distribution. Yet VLA models, typically trained onoffline datasets, lack directly observable state transitions, necessitating alearned world model to anticipate action consequences. In this work, wepropose a unified understanding-and-generation framework that lever-ages world modeling to simultaneously enable meaningful explorationand provide dense supervision. Specifically, we augment trajectory pre-diction with future RGB and depth image generation as dense worldmodeling objectives, requiring the model to learn fine-grained visual andgeometric representations that substantially enrich the planning back-bone. Beyond serving as a supervisory signal, the world model furtheracts as a source of intrinsic reward for policy exploration: its imageprediction uncertainty naturally measures a trajectory’s novelty rela-tive to the training distribution, where high uncertainty indicates out-of-distribution scenarios that, if safe, represent valuable learning op-portunities. We incorporate this exploration signal into a safety-gatedreward and optimize the policy via Group Relative Policy Optimiza-tion (GRPO). Experiments on the NAVSIM and nuScenes benchmarksdemonstrate the effectiveness of our approach, achieving a state-of-the-art PDMS score of 93.7 and an EPDMS of 88.8 on NAVSIM. The codeis available at https://zihaosheng.github.io/ExploreVLA/.