SyncVL: Synchronizing Vision ⟷ Language Using Unsupervised Adaptation

Contrastive vision–language models (VLMs) such as CLIP,EVACLIP, ImageBind, and SigLIP exhibit impressive zero-shot general-ization, yet their performance on specific data distributions can furtherbe enhanced by improving the synchronization of the vision and languagerepresentations. For this purpose, we present SyncVL, a bi-directionalVL synchronization framework that enables fully unsupervised adapta-tion of VLMs. SyncVL jointly refines visual and textual representationsthrough two lightweight transformer-based encoder-decoder synchroniz-ers (Vision-to-Text and Text-to-Vision) that progressively project VLrepresentation into a shared latent subspace without requiring labeleddata. To ensure robust synchronization, we form groups using multipleimage-views and text templates. Both synchronizers are trained using areward function motivated by group relative policy optimization whilekeeping the original VLMs frozen. Additionally, a mutual-distillationmechanism is proposed to iteratively reinforce each synchronizer. Weextensively evaluate SyncVL on 24 benchmark datasets spanning unsu-pervised VL adaptation, out-of-distribution matching, unsupervised clus-tering, cross-modal retrieval, object detection, and segmentation. Acrossall tasks, SyncVL consistently improves over state-of-the-art baselines,establishing a new standard for unsupervised adaptation of VLMs.