Self-Evolving MCP-GUI Agents via Automated Environment Generation and Experience Learning

Computer-use agents that combine GUI interaction withstructured API calls via the Model Context Protocol (MCP) show promisefor automating software tasks. However, existing approaches lack a prin-cipled understanding of how agents should balance these two modalitiesand how to enable iterative self-improvement across diverse applica-tions. We formulate MCP-GUI interplay as a unified hybrid policylearning problem where the agent learns when each modality providescomplementary advantages, and show that distillation and experienceaugmentation target fundamentally different failure modes—requiringapplication-aware mechanism selection. Built on this formulation, wepropose a self-evolving framework with a fully automatic pipelinethat orchestrates automatic environment generation and validation, tra-jectory collection, gap-driven task synthesis, and quality-filtered training—all without manual intervention. A key innovation is our experiencebank, which accumulates LLM-learned rules from trajectory comparison,enabling inference-time improvement without fine-tuning. Systematiccross-application analysis across three desktop applications revealsthat the optimal strategy depends on MCP–GUI composition and tool-chain complexity: distillation achieves 77.8% pass rate on MCP-dominanttasks (+17.8pp), while the experience bank excels on GUI-intensive tasks(+10.0pp). Code available at: https://github.com/Tiantian-H/EE-MCP