Rethinking Prototype-based Similarity Learning for Few-Shot Object Detection

Few-shot object detection aims to detect novel object cate-gories from only a few labeled examples, avoiding costly large-scale an-notation. Recent prototype-based similarity learning approaches enabletraining-free adaptation by matching query features with class proto-types. However, they suffer from two fundamental limitations: (i) classconfusion arising from inter-class similarity margin collapse, and (ii)insufficient visual cues for precise localization, as similarity scorescapture only class-level semantic affinity while providing limited spa-tial information. To address these issues, we introduce two complemen-tary components. Text-Anchored Semantic Mask (TSMa) lever-ages class-level text features as semantic anchors to identify semanti-cally aligned channels through channel-wise interaction between visualand text features. By suppressing style-induced spurious responses andemphasizing class-intrinsic signals, TSMa enlarges inter-class similar-ity margins and mitigates class confusion. We further propose Stage-Aligned Hierarchical Autoregressive Regression (SHARe), whichreformulates localization as a hierarchical autoregressive process thatprogressively refines bounding boxes across multiple stages. SHARe lever-ages the layer-wise characteristics of ViT representations by aligning fea-ture abstraction levels with regression stages: deeper layers guide earlycoarse localization, while shallower layers rich in edge and texture cuesrefine spatial details in later stages. Experiments on COCO demonstratea new state of the art, outperforming the previous best by +10.1 nAP,with extensive analysis validating each component. The code is availableat https://github.com/VisualScienceLab-KHU/ReSet.