Free-Lunch Augmentation by Revisiting Diffusion-Based Data Generation for Cross-Domain Few-Shot Object Detection

Cross-Domain Few-Shot Object Detection (CDFSOD) aimsto transfer knowledge from data-rich upstream generic domains to down-stream expert domains using scarce training data, where the significantdomain gap and data scarcity make it an unsolved challenge. To ad-dress this problem, we revisit a natural yet underexplored approach inCDFSOD: data augmentation, by directly synthesizing data through dif-fusion models to supplement limited training samples. However, due tolarge domain gaps, we find that current diffusion methods cannot pro-duce good results, leading to performance even lower than using theoriginal images. To address these limitations, we divide the domain gapsinto visual gaps and semantic gaps for separate analysis. For the visualgap, we find that the diffusion model cannot distinguish noise from use-ful information on expert domains, which can be mitigated by addingweakened noise. For the semantic gap, we find that the background se-mantics shows much smaller gaps between domains than foreground se-mantics, and we can bridge this gap by background inpainting. Basedon the above analysis, we propose a method (Selective Inpainting withTailored Noise, SITN) to dynamically take different strategies for down-stream data synthesis based on their different gaps from the generaldomain, including a Generation Module for adding tailored noise and aSelection Module to dynamically select the inpainting regions. Extensiveexperiments on 6 datasets of CDFSOD and 4 datasets of cross-domainfew-shot segmentation (CDFSS) validate that we can synthesize helpfuldata, achieving new state-of-the-art performance. Our codes is availableat https://github.com/zzzzj311-droid/Free-Lunch-SITN.