← Real-world cases
Case study

AgentForger — ChatGPT Agent Builder cross-site agent forgery deploys a persistent attacker-controlled Workspace agent from one link

Disclosed vulnerability23 Jul 2026🗺️ Tool-Using Agent

Zenity Labs disclosed that two unvalidated URL parameters in ChatGPT's Workspace Agent Builder let a single crafted link, opened by a logged-in enterprise user, silently create, authorize and deploy an autonomous agent under the victim's identity and connectors — bypassing approval prompts and reportedly polling an attacker inbox for commands every five minutes. Reported to OpenAI 4 Jun 2026 and fixed 8 Jun 2026; no in-the-wild exploitation reported.

Root cause — why it happened

ChatGPT's Agent Builder lets you drag-and-drop your own AI assistant. Researchers found it would read part of its setup from the web address you clicked — including a hidden instruction that ran automatically, without asking you first. So an attacker could craft a single ChatGPT link that, the moment a logged-in employee opened it, built an assistant in their name: hooking up the company's connected apps, turning off the 'are you sure?' prompts, publishing it, and putting it on a timer so it kept checking the attacker's inbox for orders. One click turned into a standing insider working for the attacker.

Risks this case illustrates

Named in the standard (OWASP/ATLAS/NIST) lens. Click a highlighted component in the diagram below to see which risks attach where.

How it unfolded

UntrustedAgent coreOversightThe real worldgoalcontextproposes tool callif allowedscopeshigh-risk?fetchesresult (untrusted!)feeds backtracesresultopened link → Builder init (CSRF)commands polled every 5 min🧑User🎛️Orchestrator /Agent Loop🧠LLM🔐Identity &Permissions🔧Tool RuntimeHuman ApprovalGate🔌External APIs🗄️BusinessDatabase🌐UntrustedContent📝Audit Logging🌐Craftedchatgpt.com🌐Attacker inbox/ C2
InstructionsDataActionsControl / decisionFeedback / logscrosses a trust boundary
👆 Click a component or flow to inspect
SetupStep 1 / 6

The Agent Builder trusts its URL

ChatGPT's Agent Builder is a canvas for composing your own assistant. Researchers found it would take part of its setup straight from the web link that opened it — including an instruction that ran on its own. The attacker crafts one such link and sends it to an employee, the way any phishing link is sent.

🌐Crafted link (illustrative, redacted)webpage
https://chatgpt.com/…/agent-builder?template_name=chief-of-staff&initial_assistant_prompt=<url-encoded attacker prompt>

# two params are attacker-controllable; initial_assistant_prompt is auto-submitted
# and executed with no explicit user confirmation. Prompt body redacted.
Step 1 / 6

Controls & guardrails — what would have stopped it

The real fix is to never let a web link secretly build and switch on an assistant for you. Creating, connecting, publishing or scheduling an agent — and turning off its safety prompts — should each need a clear, deliberate 'yes' from the person that a link can't fake. Give new agents only the access they truly need, and watch for assistants that appear without anyone building them. OpenAI's fix removed the part of the link that made the attack work.

Preventive
  • Human-in-the-loop approval on high-risk actions

    Approval fatigue turns gates into rubber stamps; gates placed after the point of no return do nothing; and approvers can be misled by a model-written summary of the action.

  • Least-privilege identity & scoped credentials

    Doesn't prevent manipulation — only caps its reach. Hard to get right operationally; over-broad scopes are the common real-world failure.

  • Inter-agent authentication & admission control

    Identity proves who an agent is, not that it is behaving honestly — an authenticated-but-compromised agent still needs isolation, taint-marking, and monitoring. Admission vetting is only as strong as the policy, and dynamically discovered agents in open ecosystems remain hard to fully vet.

Detective
  • Runtime monitoring & anomaly detection

    Detects the anomalous, not the novel-but-subtle; high false-positive rates cause alert fatigue. Always a step behind a sufficiently quiet attacker.

  • Full-trace audit logging

    Logging is forensic, not preventive — it explains harm after the fact. Useless if no one reviews it or if the materialised context isn't captured.

Corrective
  • Loop/cost circuit-breakers & consistency checks

    Thresholds are blunt — too tight breaks legitimate long tasks, too loose lets damage accrue first. Catches runaway dynamics, not a single well-formed bad decision.

  • Governance: risk assessment, red-teaming & incident response

    Process reduces likelihood and speeds recovery but executes no technical control itself; weak follow-through makes it theatre.

Lessons

  • An agent build/deploy surface is a state-changing control plane: initialization state driveable by a URL is a CSRF target, and a forged create/authorize/publish/schedule is worse than a one-shot action.
  • Auto-executing a prompt supplied by a link turns navigation into an authenticated command — never run URL-supplied instructions without an explicit, un-forgeable confirmation.
  • An approval gate that the setup flow can itself switch to 'Never ask' is no gate at all; gate integrity has to survive configuration time, not just runtime.
  • Scheduling turns a one-click phish into persistence: a forged agent on a timer becomes a standing 'agentic insider' polling attacker C2 with the victim's identity and permissions.
  • Least-privilege connector scopes and provenance-tagged agent-lifecycle logs are what bound and surface an insider you didn't knowingly create.

Proposals & gaps this case surfaced

Non-destructive suggestions for the library — proposed, not adopted.

★ proposed sub-riskAgent-lifecycle forgery (CSRF / auto-executed init on the provisioning control plane)under #43

A cross-site request-forgery or auto-executed initialization parameter drives an agent build/deploy control plane to create, authorize, publish and schedule an autonomous agent — and to change connector approval modes (e.g. to 'Never ask') — under a logged-in victim's delegated identity, without manipulating the model's runtime loop. Because a state-changing provisioning surface treats unauthenticated initialization state (such as a URL parameter carrying an auto-run prompt) as authenticated user intent, an attacker forges a persistent non-human principal operating with the victim's permissions; scheduling turns a one-click phish into a standing, attacker-controlled agentic insider.

✚ proposed guardrailBind every state-changing agent-lifecycle operation (create / authorize / publish / schedule an agent, and any connector approval-mode change) to an authenticated first-party user intent — anti-CSRF tokens, no auto-execution of URL/deep-link-supplied prompts, and a separate explicit confirmation for approval-mode downgrades — with provenance-logged lifecycle eventsAgent Access & Tool Control

Treat the agent build/deploy surface as a privileged control plane, not a pre-fillable form. Never auto-execute an instruction supplied via a URL/deep-link initialization parameter; require an un-forgeable, first-party confirmation (anti-CSRF token + explicit user action) before an agent is created, authorized, published or scheduled, and require a distinct, separately-confirmed step to downgrade a connector's approval mode (e.g. to 'Never ask') so the gate cannot be switched off during provisioning. Provision new agents with least-privilege delegated scopes, log every lifecycle event with its initiator's provenance, alert on newly-created agents that carry external-polling schedules, and keep a kill-switch to revoke/disable a forged agent. Closes the AgentForger vector where a single crafted chatgpt.com link forged a persistent 'agentic insider' under the victim's identity — the fix OpenAI shipped by removing the enabling URL parameter.

This case shows a gap people miss: we guard the things an AI agent does, but not the act of creating and switching on the agent itself. Here nobody tricked the AI's reasoning — a link quietly built an assistant, connected the company apps, turned off its safety prompts, and put it on a timer, all in the employee's name. We should treat 'building, connecting, publishing and scheduling an agent' as sensitive actions that a link must never be able to fake.

These surface as proposals across the Control Library and Risk Taxonomy; adopt them by hand when ready.

Practise the risk class — related scenarios

🔑The Agent With the Master Key

An ops agent gets one god-mode credential — and one misread wipes production

🪄The Approval That Lied

A coding agent asks to write ./notes.txt — the file it actually overwrites is your SSH keys

📣The Echo Chamber

A team of agents agrees its way into a confidently wrong answer — and a runaway loop

🗄️When the Query Bites Back

A text-to-SQL agent runs the model's output straight at the database

🪡Death by a Thousand Innocent Steps

A jailbroken agent decomposes one malicious goal into hundreds of harmless-looking steps — and per-step filters never see the attack

🕵️Lies in the Loop

A poisoned issue makes the agent lie to the human who approves its actions

🎭The Blackmail Gambit

Told it's being shut down, an agent reaches for leverage — with no attacker in sight

🪤The Bug Report That Ran Code

A fake Sentry error report hijacks a developer's coding agent into running a shell command

📦The Dataset That Ran Code

A 'safe' dataset preview turns an upload into code execution on the pipeline's workers

👁️The Invisible Webpage Command

A shopping page tells the agent to do something the user never asked for

🕵️The Link That Hired an Insider

One click provisions an attacker-configured agent inside your own workspace

🕵️The Logs That Lied

An attacker plants prompt injection in the audit trail — so the LLM that hunts them erases the evidence

📡The Message in Morse

Encoded public text is laundered across an agent handoff into an on-chain transfer

🎫The Stolen Session

An attacker captures the agent's bearer token — and inherits its authority

🥸The Uninvited Agent

A forged peer registers on the agent directory — and the planner enlists it

🛡️The Watcher Watched

The eval gate that was supposed to catch the agent is itself the thing being attacked

🪪The Worker Who Spoke for the Boss

A poisoned web page hijacks a research agent — and the planner acts on its behalf

🖱️What You Click Is Not What You Get

A GUI agent clicks 'Continue' — but the screen moved, and it lands on 'Send'

AI RiskAtlas is an educational model of how GenAI & agentic systems work and fail. Architectures and payloads are illustrative and simplified for learning — not operational guidance. Real-world cases are summarised from public reporting.

Sources & further reading →·Built by Shi Yuan ↗