AnyGround3D: Towards Grounding Any 3D Object in the Wild via 2D-to-3D Lifting

3D visual grounding aims to localize language-referred ob-jects in 3D scenes. However, existing methods rely on limited annotateddatasets under controlled environments and thus struggle to generalizein the wild. Therefore, the scarcity of scalable 3D grounding supervisionremains a major challenge. In this paper, we propose AnyGround3D,a scalable framework that learns to ground arbitrary 3D objects in thewild via 2D-to-3D lifting. Instead of requiring manual 3D annotations,our approach lifts diverse Internet images into structured 3D supervisionthrough multi-level 2D-to-3D lifting. First, we perform scene-level 2D-to-3D lifting to reconstruct metric point clouds from monocular images,providing geometric context. Second, we conduct object-level 2D-to-3Dlifting to recover complete 3D object representations with fine-gained 3Dbounding boxes and neural language descriptions. To further enhancethe representation robustness and open-vocabulary generalization of thepoint encoder, we introduce feature-level lifting that distills semantic rep-resentations from 2D foundation models into the 3D encoder. Together,these lifting processes enable large-scale synthetic 3D grounding supervi-sion across diverse real-world scenes. Extensive experiments demonstrateimproved in-the-wild generalization across multiple 3D grounding bench-marks, without relying on human-annotated 3D grounding data.