Training-free Discriminative Patch Mining for Robust Few-Shot Recognition with CLIP

Few-shot classification performance of Contrastive Language-Image Pre-training (CLIP) varies widely across datasets, especially whenclass names provide weak or ambiguous semantic priors. This issue is alle-viated in vision-only fine-grained methods, as they identify discriminativelocal features but require training on the full dataset. We thus introduce atraining-free approach that integrates discriminative patches into CLIPto reduce reliance on textual prompts. Our method identifies patcheswith high intra-class consistency and low inter-class ambiguity, forminga Class-Discriminative Patch Set (CDPS) for each category. Using CDPS,we enhance the recognition ability of CLIP through a hybrid classifiercombining global image-text alignment with local patch-based similarity.Extensive experiments on diverse benchmarks show that CDPS injectsfine-grained discriminatory power into CLIP and yields more robust few-shot recognition performance against ambiguous class names.