VLOD-TTA: Test-Time Adaptation of Vision-Language Object Detectors

Vision-language object detectors (VLODs) such as YOLO-World and Grounding DINO exhibit strong zero-shot generalization, buttheir performance degrades under distribution shift. Test-time adapta-tion (TTA) offers a practical way to adapt models online using onlyunlabeled target data. However, despite substantial progress in TTAfor vision-language classification, TTA for VLODs remains largely unex-plored. The only prior method relies on a mean-teacher framework thatintroduces significant latency and memory overhead. To this end, weintroduce VLOD-TTA, a TTA method that leverages dense proposaloverlap and image-conditioned prompts to adapt VLODs with low addi-tional overhead. VLOD-TTA combines (i) an IoU-weighted entropy ob-jective that emphasizes spatially coherent proposal clusters and mitigatesconfirmation bias from isolated boxes, and (ii) image-conditioned promptselection that ranks prompts by image-level compatibility and aggregatesthe most informative prompt scores for detection. Our experiments acrossdiverse distribution shifts, including artistic domains, adverse drivingconditions, low-light imagery, and common corruptions, indicate thatVLOD-TTA consistently outperforms standard TTA baselines and theprior state-of-the-art method using YOLO-World and Grounding DINO.Code: https://github.com/imatif17/VLOD-TTA