Restoring Linguistic Grounding in VLA Models via Train-Free Attention Recalibration

Vision-Language-Action (VLA) models enable robots to per-form manipulation tasks directly from natural language instructions andare increasingly viewed as a foundation for generalist robotic policies.However, their reliability under Out-Of-Distribution (OOD) instructionsremains underexplored. In this paper, we reveal a critical failure modein which VLA policies continue executing visually plausible actions evenwhen the language instruction contradicts the scene. We refer to this phe-nomenon as linguistic blindness, where VLA policies prioritize visualpriors over instruction semantics during action generation. To systemati-cally analyze this issue, we introduce ICBench, a diagnostic benchmarkconstructed from the LIBERO dataset that probes language–action cou-pling by injecting controlled OOD instruction contradictions while keep-ing the visual environment unchanged. Evaluations on three represen-tative VLA architectures, including π0 , π0.5 , and OpenVLA-OFT, showthat these models frequently succeed at tasks despite logically impossi-ble instructions, revealing a strong visual bias in action generation. Tomitigate this issue, we propose Instruction-Guided Attention Re-calibration (IGAR), a train-free inference-time mechanism that rebal-ances attention distributions to restore the influence of language instruc-tions. IGAR operates without retraining or architectural modificationand can be directly applied to existing VLA models. Experiments across30 LIBERO tasks demonstrate that IGAR substantially reduces erro-neous execution under OOD contradictory instructions while preservingbaseline task performance. We additionally validate the approach on areal Franka robotic arm, where IGAR effectively prevents manipulationtriggered by inconsistent instructions.