The 3D Mirage: Probing and Taming 3D Hallucinations

Monocular depth foundation models achieve remarkable gen-eralization by learning large-scale semantic priors, but this creates a criti-cal vulnerability: they hallucinate illusory 3D structures from planar/low-curvature but perceptually ambiguous inputs. We term this failure the3D Mirage. This paper introduces a novel end-to-end framework toprobe, score, and tame this under-quantified safety risk in monoculardepth under context variation. To probe, we present 3D-Mirage, thefirst benchmark to combine context variation and precise annotation forreal-world illusions with real object exclusions, multi-surface support;purpose-built to stress-test monocular depth on real-world illusions. Toscore, we propose a second-order magnitude-based evaluation with twometrics: the Deviation Composite Score (DCS) for high second-order 3D structure and the Confusion Composite Score (CCS) forcontextual instability. To tame this failure, we introduce GroundedSelf-Distillation, a parameter-efficient strategy on Depth-Anything-V2baseline that surgically targets and resolves hallucination on illusion ROIswhile preserving background knowledge, avoiding catastrophic forgetting.Our work provides an innovative pipeline for diagnosing and addressingthis phenomenon, urging a necessary shift in the evaluation of MDE frompixel-wise accuracy to structural and contextual robustness.Code: https://github.com/hdnndh/The- 3D- Mirage- Probing- and-Taming-3D-HallucinationsDataset: https://huggingface.co/datasets/3dmirage/3D-Mirage