ReconSplat: Generalizable 3D Scene Reconstruction Beyond Observed Views

We introduce ReconSplat, a feed-forward model for 3D scenereconstruction that aims to address the longstanding trade-off betweenplausible view generation for unobserved regions and geometric consis-tency, providing both geometrically aligned novel views and sharp depthestimates. Our approach builds on 3D Gaussian splatting (3DGS) as anintermediate differentiable scene representation and integrates it with amulti-view latent diffusion model (MV-LDM) trained to act simultane-ously as a refiner and an inpainter for appearance and scene geometry.We enforce geometric consistency by guiding the diffusion process withvariational 3D latent features for appearance and geometry, encoded bythe feed-forward 3DGS representation and rasterized to 2D latent space.ReconSplat produces both photorealistic novel views and accurate depthmaps on real-world benchmarks, RealEstate10K and DL3DV-10K, out-performing existing methods in challenging extrapolation setups. No-tably, ReconSplat allows the extrapolation of unseen and challengingviewpoints jointly with coherent and precise scene geometry. Code andadditional visual results are available on our project page.