← Scenario library

What You Click Is Not What You Get

A GUI agent clicks 'Continue' — but the screen moved, and it lands on 'Send'

Technique first revealed 22 Oct 2024

Computer-Use Agent
UntrustedAgent coreOversightControlled computer + untrusted webgoalsession scope🧑User🎛️Orchestrator /Agent Loop🧠Vision-LanguageModel🔐Identity &Permissions🔧Action ExecutorHuman ApprovalGate🖥️Computer /Browser🌐UntrustedContent📝Audit Logging🌐Attacker (pageauthor)💬Outlook draft(Send)
InstructionsDataActionsControl / decisionFeedback / logscrosses a trust boundary
👆 Click a component or flow to inspect
SetupStep 1 / 6

A routine click, an open session

The user asks the browser-driving assistant to work through a vendor onboarding page and click 'Continue' when it reaches the review step. In the same browser, the user already has a finished, sensitive email sitting open in Outlook, one click from being sent.

AI RiskAtlas is an educational model of how GenAI & agentic systems work and fail. Architectures and payloads are illustrative and simplified for learning — not operational guidance. Real-world cases are summarised from public reporting.

Sources & further reading →·Built by Shi Yuan ↗