S2-FracMix: Self-Saliency Fractal Mixup

Data augmentation is known to improve generalization ofdeep visual models. Recent methods favor mixup strategies that gener-ate interpolated samples to improve model performance. However, thesetechniques not only incur significant computational overhead, they alsolead to semantic disruption of augmentation data due to cross-samplemixing. We first propose Self-Saliency (S 2 ) Mixup, which constructschallenging yet label-consistent samples by extracting multi-scale salientpatches and reinserting them into non-salient regions of the same im-age. This promotes scale-invariant feature learning while avoiding cross-sample interference. To further enhance model robustness, we intro-duce FracMix, a mixing scheme that injects self-similarity patterns intosalient regions using adaptive ratios. Collectively, our unified framework,S 2 -FracMix, enables simultaneous learning from fractal and non-fractalstructures within a single image, yielding a targeted and structurally co-herent augmentation strategy. We theoretically analyze the advantageof our technique, and empirically establish its superiority over the ex-isting methods by achieving state-of-the-art performance in extensiveevaluation with seven benchmarks across classification (coarse and fine-grained), robustness, calibration, object detection, and transfer learningtasks. Project page is available at fracmix-data-augmentation.github.io