Efficient RGB-T Object Detection via Sparse Cross-Modality Fusion

RGB-T detectors leverage the complementary strengths ofvisible and thermal infrared modalities, achieving robust performanceunder challenging conditions. Many of them resort to heavy dual back-bones and exhaustive cross-modality fusion across the entire image, lead-ing to impractically high computational costs. We observe that most im-age regions are smooth backgrounds (e.g., sky, ground) that can be easilyhandled by lightweight single-modality models. In light of this observa-tion, we propose a sparse fusion mechanism for efficient RGB-T detec-tion: first rapidly scanning the image to identify the proposals and thencarefully examining the remaining sparse proposals via feature fusion.We propose a two-stage framework to instantiate this mechanism, whichperforms detection in two stages: 1) a lightweight and modality-specificdetection stage that produces high-recall RoIs, and 2) a fusion-driven ex-amination and refinement stage that filters out the false positives and re-fines the bounding boxes. This design enables the detector to adaptivelyallocate more computational resources to the potential foregrounds, im-proving the efficiency while ensuring detection accuracy. Extensive ex-periments show that our method achieves competitive performance withsubstantially fewer parameters and lower cost, while maintaining strongscalability to high-resolution images.