DualResPS: Dual-Resolution Photometric Stereo Using a Frame-Event Hybrid Camera

As an important 3D sensing method, photometric stereo estimates surface normals from multiple photos taken under varying lighting. The surface normal quality highly relies on large number of input images, which requires long capturing time, high data burden and limits high-speed applications for photometric stereo. Recently, event-based photometric stereo has emerged as an efficient and rapid approach, but the low resolution of event sensors and the noise of event signals limit its high-quality application. In this paper, we propose DualResPS, a novel photometric stereo pipeline utilizing a frame-event hybrid camera. We design a lighting and capturing strategy tailored to a hybrid-camera setup to utilize observations of different spatial and temporal resolutions. The algorithm explicitly models non-ideal factors including shadow effects and specular reflection to achieve high-resolution, high-quality normal reconstruction. Our proposed DualResPS is validated on both semi-real datasets from the DiLiGenT, DiLiGenT-Pi, and real captured data. The results demonstrate that DualResPS surpasses its frame-based counterpart with 18.1% data bandwidth.