PanoSAM2: Lightweight Distortion- and Memory-aware Adaptions of SAM2 for 360 Video Object Segmentation

360 video object segmentation (360VOS) aims to predicttemporally-consistent masks in 360 videos, offering full-scene coverage,benefiting applications, such as VR/AR and embodied AI. Learning360VOS model is nontrivial due to the lack of high-quality labeled dataset.Recently, Segment Anything Models (SAMs), especially SAM2 – withits design of memory module – shows strong, promptable VOS capabil-ity. However, directly using SAM2 for 360VOS yields implausible resultsas 360 videos suffer from the projection distortion, semantic inconsis-tency of left-right sides, and sparse object mask information in SAM2’smemory. To this end, we propose PanoSAM2, a novel 360VOS frame-work based on our lightweight distortion- and memory-aware adaptationstrategies of SAM2 to achieve reliable 360VOS while retaining SAM2’suser-friendly prompting design. Concretely, to tackle the projection dis-tortion and semantic inconsistency issues, we propose a Pano-AwareDecoder with seam-consistent receptive fields and iterative distortionrefinement to maintain continuity across the 0°/360° boundary. Mean-while, a Distortion-Guided Mask Loss is introduced to weight pixelsby distortion magnitude, stressing stretched regions and boundaries. Toaddress the object sparsity issue, we propose a Long–Short Mem-ory Module to maintain a compact long-term object pointer to re-instantiate and align short-term memories, thereby enhancing temporalcoherence. Extensive experiments show that PanoSAM2 yields substan-tial gains over SAM2: +5.6 on 360VOTS and +6.7 on PanoVOS, show-ing the effectiveness of our method. Code is available at https://github.com/Eric-Bumaro/PanoSAM2.