MV2GF: Multi-view Pedestrian Detection with a Visual Geometric Foundation Model

Multi-View Pedestrian Detection (MVPD) aims to detectpedestrians in the form of a bird’s eye view map from multi-view images.Recent MVPD methods adopt a unified framework that projects 2D im-age features into a 3D world space and aggregates them into a singlefeature. Although they are effective, they struggle to generalize to un-seen camera configurations during training due to two main issues. First,they are difficult to capture accurate visual geometry across views in un-seen camera configurations. Second, they make detection models highlydependent on distortion patterns during training arising from their im-age feature projection. To address these, we leverage a visual geometricfoundation model and propose MV2GF. This foundation model has ex-hibited strong generalization in capturing visual geometry across viewsand predicting accurate 3D attributes in diverse camera configurations.MV2GF fuses task-specific features with general-purpose geometric fea-tures extracted by the foundation model to effectively capture the visualgeometry even in unseen camera configurations. Furthermore, MV2GFprojects each pixel in the image features to an appropriate 3D locationusing 3D pointmaps predicted by the foundation model, preventing thedetection model from depending on distortion patterns during training.Our experiments demonstrate the effectiveness of leveraging a visual geo-metric foundation model for MVPD and that MV2GF generalizes betterthan existing methods.