VLMSysTrojan: Stealthy System-Aware Backdoor Attacks Against Vision-Language Models

Vision–Language Models (VLMs) are increasingly deployedin real-world applications. To ensure efficient inference, these deploy-ments typically rely on specialized system kernels (e.g., CUDA or cuDNN)for acceleration. However, existing system-level research on VLMs has fo-cused primarily on improving kernel efficiency and performance, whilelargely overlooking their potential impact on model security and ro-bustness. In particular, the security implications of inconsistencies be-tween training and inference kernels remain underexplored. To addressthis gap, we present the first systematic study revealing a new classof vulnerabilities: system-level backdoor attacks that exploit floating-point inconsistencies across training and inference kernels. We introduceVLMSysTrojan, a red-teaming framework that constructs models behav-ing benignly under standard training kernels but exhibiting backdooredbehavior when executed on specific target inference kernels. Specifically,VLMSysTrojan produces models whose backdoor triggers remain inactiveunder standard kernels and successfully evade three state-of-the-art back-door detection methods. Yet, when deployed on a target kernel, kernel-specific floating-point behavior activates the backdoor. Empirically, thecompromised models achieve a 99% attack success rate on triggered in-puts while maintaining normal accuracy on clean inputs. Moreover, theattack generalizes across multiple kernel configurations, exposing a previ-ously unrecognized risk in VLM deployment and highlighting the urgentneed for kernel-aware robustness analysis techniques.