Multi-view Multi-vehicle Driving Dataset for Novel View Synthesis

Di!erentiable rendering has advanced novel view synthesis(NVS), yet applying it to real-world driving remains di"cult due tosparse capture viewpoints, dynamic objects, and limited multi-trajectorydata. We introduce the Multi-View Multi-Vehicle (MV2 ) datasetand benchmark for evaluating NVS models under large viewpoint changesin dynamic urban scenes. MV2 features synchronized captures from a car,scooter, and drone, each following distinct yet synchronized trajectories.Training NVS methods on one vehicle’s camera stream and testing on an-other enables evaluation under substantially larger viewpoint variationsthan existing single-trajectory datasets. All sequences are registered viaStructure-from-Motion and camera poses verified using manual pixel-level correspondence annotations, yielding 50 high-quality scenes with12000 images. Benchmarking recent NVS and camera pose estimationmethods shows that NVS performance degrades with increasing view-point disparity, and that feed-forward pose estimators notably lag behindoptimization-based approaches, highlighting MV2 as a rigorous testbedfor NVS in driving. The dataset, benchmark protocol, and project re-sources are available at https://mv2-dataset.github.io/.