GaussianLens: Localized High-Resolution Reconstruction via On-Demand Gaussian Densification

We perceive our surrounding environments with an active fo-cus, paying more attention to regions of interest, such as the shelf labelsin a grocery store or a family photo on the wall. When it comes to scenereconstruction, this human perception trait calls for spatially varyingdegrees of detail ready for closer inspection in critical regions, prefer-ably reconstructed on demand as users shift their focus. While recentapproaches in 3D Gaussian Splatting (3DGS) can achieve fast, general-izable scene reconstruction from sparse views, their uniform resolutionoutput leads to high computational costs, making them unscalable tohigh-resolution training. As a result, they cannot leverage available imagecaptures at their original high resolution for detail reconstruction. Per-scene optimization methods reconstruct finer details with heuristic-basedadaptive density control, yet require dense observations and lengthy of-fline optimization. To bridge the gap between the prohibitive cost of high-resolution holistic reconstructions and the user needs for localized finedetails, we propose the problem of localized high-resolution reconstruc-tion through on-demand generalizable Gaussian densification. Given aninitial low-resolution 3DGS reconstruction, the goal is to learn a gener-alizable network that densifies the reconstruction to capture fine detailsin a user-specified local region of interest (RoI), based on sparse high-resolution observations of the RoI. This formulation avoids the high costand redundancy of uniformly high-resolution reconstructions and enablesthe full leverage of high-resolution observations in critical regions. To ad-dress the problem, we propose GaussianLens, a feed-forward densifica-tion framework that fuses multi-modal information from the initial 3DGSand multi-view images. We further propose a pixel-guided densificationmechanism that effectively captures details under significant resolutionincreases. Experiments demonstrate our method’s superior performancein local high-fidelity detail reconstruction and strong scalability to im-ages of up to 1024 × 1024 resolution.