Seeing to Ground: Visual Attention for Hallucination-Resilient MDLLMs

Multimodal Diffusion Large Language Models (MDLLMs)achieve high-concurrency generation through parallel masked decoding,yet the architectures remain prone to multimodal hallucinations. Thisstructural vulnerability stems from an algorithmic flaw: the decoderranks candidate tokens based on textual likelihood without verifyinglocalized visual support. We establish that this language-only rankinginduces an objective mismatch, where language probability mass acts asa misspecified proxy for the intended multimodal task. Consequently, wereinterpret hallucination as a localized optimization error, a phenomenonwhere the decoder exploits language shortcuts to maximize a proxy scoreat the expense of visual grounding. To address this objective mismatch,we introduce VISAGE, a training-free decoding framework that cali-brates the objective at inference time. VISAGE estimates the proxydiscrepancy by quantifying the spatial entropy of cross-attention distri-butions. By enforcing a localization consensus across attention heads, themethod penalizes spatially uniform distributions and re-ranks token com-mitments to favor visually grounded outcomes. We provide an analyti-cal stability guarantee establishing that VISAGE maintains a boundedobjective loss under estimation error. Evaluations across hallucination-sensitive and general-purpose benchmarks demonstrate the robustnessof the framework, yielding relative gains of 8.59% on MMMU-val and7.75% on HallusionBench.