AnchorGUI: Asymmetric Memory for Dual-Scale Learning in GUI Navigation

Vision-Language Models (VLMs) enable autonomous GUInavigation, but agents still struggle to process and learn from dense,continuous visual histories. This bottleneck hinders both immediate errorcorrection within a single episode (intra-trial) and experience distilla-tion across multiple attempts (cross-trial). We trace these challenges toan empirical informational asymmetry in GUI navigation: while expectedtransitions can often be compressed into lightweight textual summaries,unexpected outcomes benefit from preserved screenshots as causal evi-dence for accurate diagnosis. Building on this insight, we propose An-chorGUI, a unified framework driven by the Cognitive State Anchor(CSA). The CSA acts as a per-step primitive that actively comparesexpected and observed transitions, converting passive multimodal tra-jectories into explicit prediction-error signals. These signals orchestratea dual-scale learning mechanism via an asymmetric memory. Forintra-trial correction, a sliding window selectively retains visual evidencefor detected mismatches, providing immediate, visually-grounded feed-back. For cross-trial distillation, this asymmetric memory focuses thecomputationally expensive credit assignment search space on likely fail-ure steps. Experiments across four benchmarks validate the effectivenessof our approach. On AndroidWorld, AnchorGUI achieves a 57.3% successrate with a 2.4× token reduction per step. Furthermore, cross-trial distil-lation reaches 69.2% success (+11.9% gain), significantly outperformingstandard reflection methods while maintaining sub-linear context scaling.