Filterless Snapshot Hyperspectral Imaging using Guided Patch Diffusion

We consider the problem of reconstructing a H × W × 31 hy-perspectral image from a H × W grayscale snapshot measurement thatis captured using only a single diffractive lens and a filterless panchro-matic photosensor. This problem is severely ill-posed, but we presenta model that produces high-quality results in simulation and experi-ment. We make efficient use of limited training data by creating a con-ditional denoising diffusion model that operates on small patches in ashift-invariant manner. During inference, we synchronize per-patch hy-perspectral predictions using guidance by physical consistency with thesystem’s optical point spread function. Our experiments reveal that thepatch size can be as small as the point spread function, with local opticalcues being the main source of information about complete spectra. Also,by drawing multiple samples, our model provides per-pixel uncertaintyestimates that strongly correlate with reconstruction error.