Improving Adversarial Robustness via Activation Amplification and Attenuation

The existence of adversarial attacks is often attributed tothe presence of non-robust features in neural networks. While prior de-fenses reduce their impact via pruning, masking, or feature recalibra-tion, we instead propose to jointly learn to amplify and attenuate thesesignals through a simple activation scaling mechanism. To this end,we introduce Activation Amplification and Attenuation (A3), alightweight plug-in module that enhances adversarial robustness withminimal modifications of the activations. A3 dynamically rescales theactivations using a learnable mask and a scaling factor derived from theoriginal activation magnitudes. The influence of adversarial perturba-tions can be amplified or attenuated using the same learnable parametersby simply flipping the sign of the scaling operation. The amplified signalsserve as negative references to construct novel contrastive and rankingloss functions. Experimental analysis shows that learning to degrade thepredictions in amplification mode simultaneously improves adversarialrobustness in attenuation mode. Moreover, A3 relies on only a smallnumber of learnable parameters, with most of its behavior being deter-mined by the scaling mechanism rather than additional network capac-ity. Extensive experiments demonstrate that integrating A3 into differentbackbones, datasets, and training methods consistently improves adver-sarial robustness while introducing negligible computational and mem-ory overhead compared to existing plug-in modules. Code is available at:https://github.com/tgoncalv/A3.