New Memory-Driven Framework Enhances Adaptive GUI Agents

Global: New Memory-Driven Framework Enhances Adaptive GUI Agents

Researchers from the University of Science and Technology, led by Libo Sun, introduced a novel adaptive agent framework called MAGNET on Jan. 27, 2026. The system targets mobile graphical user interface (GUI) agents that rely on large foundation models, aiming to maintain performance despite frequent UI updates that alter visual layouts while preserving underlying functional semantics.

Dual-Level Memory Architecture

MAGNET incorporates a dual-level memory structure. The stationary memory links a wide range of visual features to stable functional semantics, providing robust grounding for actions across varied interfaces. Complementing this, procedural memory captures enduring task intents, enabling the agent to follow consistent workflows even when screen elements shift.

Dynamic Memory Evolution Mechanism

The framework employs a dynamic memory evolution process that continuously refines both memory components. By prioritizing knowledge that is accessed frequently, the system adapts its internal representations to reflect the most relevant visual‑semantic mappings and procedural patterns.

Benchmark Evaluation

Online testing on the AndroidWorld benchmark demonstrated substantial performance gains over existing baseline agents. Offline assessments further confirmed that MAGNET retains its advantages under distribution shifts, suggesting resilience to the evolving software environments typical of mobile applications.

Implications for Future GUI Agents

The findings indicate that leveraging invariant functional structures across interface changes can improve both the accuracy and generalization of autonomous GUI agents. This approach may inform the design of more reliable AI assistants that operate across diverse and rapidly updating mobile platforms.

Next Steps and Availability

The authors plan to extend the framework to additional operating systems and explore integration with other large‑scale foundation models. The full paper, including methodological details and experimental results, is accessible via the arXiv repository.

This report is based on information from arXiv, licensed under Academic Preprint / Open Access. Based on the abstract of the research paper. Full text available via ArXiv.