๐Ÿ”AI RiskAtlas
โ† All systems

Computer-Use Agent

An agent that drives a real browser or desktop by sight

Architecture introduced 22 Oct 2024

Instead of calling neat software functions, this assistant uses a computer the way you do: it looks at the screen, then moves the mouse, clicks, and types. That makes it powerful โ€” it can use any app โ€” but it now 'reads' whatever is on screen, including web pages written by strangers.

UntrustedAgent coreOversightControlled computer + untrusted webgoal๐Ÿง‘User๐ŸŽ›๏ธOrchestrator /Agent Loop๐Ÿง Vision-LanguageModel๐Ÿ”Identity &Permissions๐Ÿ”งAction Executorโœ‹Human ApprovalGate๐Ÿ–ฅ๏ธComputer /Browser๐ŸŒUntrustedContent๐Ÿ“Audit Logging
InstructionsDataActionsControl / decisionFeedback / logs
๐Ÿ‘† Click any component in the diagram to inspect its risks & defenses

Follow a request ยท step 1 of 6

You give the assistant a goal it has to do on a real computer โ€” 'book the cheapest flight on this travel site'.

Scenarios on this architecture

AI RiskAtlas is an educational model of how GenAI & agentic systems work and fail. Architectures and payloads are illustrative and simplified for learning โ€” not operational guidance. Real-world cases are summarised from public reporting.

Sources & further reading โ†’ยทBuilt by Shi Yuan โ†—