SegPAR: Class-Centric Decision-Based Sparse Attack for Semantic Segmentation

Despite the practical relevance of sparse decision-based black-box threats, they have received limited attention in semantic segmenta-tion. To bridge this gap, we adapt the most representative decision-basedblack-box sparse attacks from the classification domain to serve as base-lines, establishing a rigorous benchmark for this underexplored setting.In this context, we demonstrate that one of the existing methods suf-fers from severe query inefficiency due to its image-centric pixel accu-mulation, which rapidly exhausts query budgets across the vast imagespace. To overcome this, we propose SegPAR, a novel decision-basedframework that shifts to a class-centric exploration paradigm. Further-more, to eliminate the misleading feedback generated by standard de-cision rewards during pixel accumulation, we introduce a novel discrep-ancy reward. Extensive experiments show that SegPAR significantly out-performs black-box baselines in sparsity efficiency and MIoU reduction,while remaining competitive with white-box sparse attacks. Code is avail-able at https://github.com/KAU-QuantumAILab/SegPAR.