A Classifier-Agnostic Zero-Shot Adversarial Attack Detection via CLIP

Adversarial attacks pose a challenge to the reliability of deeplearning models, motivating effective detection methods. Existing tech-niques often rely on attack-specific assumptions, access to adversarialsamples, or knowledge of the underlying classifier (white-box). We pro-pose A4 D (Attack- and Architecture-Agnostic Adversarial Detector), acompletely black-box, zero-shot adversarial attack detection frameworkthat utilizes prompt-based similarity scores derived from CLIP. To thebest of our knowledge this is the first attempt to utilize CLIP for sucha task. The method is based on two key observations: (i) CLIP is sen-sitive even to small imperceptible non-semantic perturbations; (ii) Theshift in CLIP embedding space is not arbitrary and can be used as arobust attack indicator. Experiments across multiple attacks, datasetsand classifiers validate that A4 D achieves SOTA detection results in theattack-agnostic and classifier-agnostic setting.