MorphJEPA: Morphology-Aware Latent Prediction for Hyperspectral Images

We study learning from limited supervision (LLS) for hyperspectral imagery (HSI): learning transferable representations from abundant unlabeled HSIs in the target sensing domain and then adapting with only a handful of labels per class. Joint-Embedding Predictive Architectures (JEPAs) are appealing for LLS because they learn via latent prediction rather than pixel reconstruction; however, they remain unexplored for HSI. Generic masking or cropping is ill-suited to HSI: band masking corrupts high-dimensional spectra, spatial masking removes entire pixel-level semantics, and dense spatial–spectral extraction can yield highly correlated, degenerate prediction targets. We propose MorphJEPA, the first JEPA tailored to HSI, which uses morphology as the pretext signal by predicting embeddings of pixel-aligned targets generated by per-band morphological opening/closing. This induces a directional structural bottleneck that preserves core topology while suppressing high-frequency nuisances. MorphJEPA couples this objective with a compact disentangled spatial–spectral encoder fused by crossattention and a context-anchored Sketched Isotropic Gaussian Regularizer (SIGReg) to prevent collapse and maintain a well-conditioned latent manifold. Extensive 1/5/10-shot evaluations show consistent gains in intra-scene settings (Houston 2013, Trento) and under cross-scene temporal/spatial shifts (Houston 2013 → 2018, HyRank) for both zero-shot transfer and low-shot adaptation. Project page: https://github.com/ amartya-ray/MorphJEPA.