BeTTER: Diagnose the Illusion of Embodied Reasoning in Vision-Language-Action Models

Recent Vision-Language-Action (VLA) models report im-pressive success rates on robotic benchmarks, but whether these scoresreflect genuine embodied reasoning remains questionable. To addressthis gap, we introduce BeTTER, a diagnostic Benchmark for TestingTrue Embodied Reasoning. By applying targeted causal interventions(e.g., spatial layout shifts and temporal extrapolation) under strict kine-matic isolation, BeTTER decouples high-level cognitive failures fromlow-level execution errors. Systematic evaluations reveal that state-of-the-art VLAs break down under these interventions, exhibiting lexical-kinematic shortcuts, semantic feature collapse, behavioral inertia, andcausal state-tracking failures. Our analysis traces these failures to twocoupled sources: real-time deployment constraints, such as capacity com-pression and myopic perception, which weaken semantic representations,and behavioral cloning, which can amplify predictive but non-causal cor-relations into shortcut policies under static training distributions. Wefurther show that highly static evaluations mask these defects by per-mitting overfitting to sensorimotor priors. Real-world robotic validationconfirms that these failures are not simulation artifacts, highlighting theneed for future VLA paradigms to preserve semantic and causal reason-ing while maintaining efficient continuous control. Code and benchmark:https://github.com/BeingBeyond/BeTTER.