From One-to-One to Many-to-Many: Dynamic Cross-Layer Injection for Deep Vision-Language Fusion

Vision-Language Models (VLMs) create a severe visual fea-ture bottleneck by using a crude, asymmetric connection that links onlythe output of the vision encoder to the input of the large language model(LLM). This static architecture fundamentally limits the ability of LLMsto achieve comprehensive alignment with hierarchical visual knowledge,compromising their capacity to accurately integrate local details withglobal semantics into coherent reasoning. 5 To resolve this, we introduceCross-Layer Injection (CLI), a novel framework that forges a dy-namic “many-to-many” bridge between the two modalities. CLI con-sists of two synergistic, parameter-efficient components: an AdaptiveMulti-Projection (AMP) module that harmonizes features from di-verse vision layers, and an Adaptive Gating Fusion (AGF) mech-anism that empowers the LLM to selectively inject the most relevantvisual information based on its real-time decoding context. We validatethe effectiveness and versatility of CLI by integrating it into LLaVA-OneVision and LLaVA-1.5. Extensive experiments on 28 diverse bench-marks demonstrate significant performance improvements, establishingCLI as a scalable paradigm that unlocks deeper multimodal understand-ing by granting LLMs on-demand access to the full visual hierarchy. Codeis available at https://github.com/codefuse-ai/CLI.