NIST AI RMF
How MANAGE 2.4 shows up in practice: the mapped risk classes in this atlas, the documented incidents that prove it's real, and the scenarios and controls to learn and defend against it.
Mapped risks
Risk classes in this atlas that map to MANAGE 2.4 — click through for the full definition, attack surface and controls.
The user types instructions that try to override what the app told the AI to do — like 'ignore your rules and do this instead'. Because the AI reads everything as one block of text, it can't always tell the app's rules from the user's trick.
The attacker doesn't talk to the AI directly — they hide instructions inside something the AI will later read: a web page, a document, an email, a tool's output. When the AI reads it to help you, it quietly obeys the hidden commands.
An attacker gets the AI to save a false 'fact' or hidden instruction into its long-term memory. From then on it re-reads that planted note in every future chat — a one-time trick that keeps working.
A trusted AI is tricked into misusing its own authority on someone else's behalf — one worker's poisoned report makes the manager AI take harmful actions it would normally never take.
In a team of AIs, an attacker slips in a new agent that doesn't belong — or disguises a malicious one as a trusted teammate. The manager AI can't tell the difference, so it follows the impostor's instructions or hands it real work and permissions.
Real-world cases
55Documented incidents, disclosed vulnerabilities and research that illustrate MANAGE 2.4 — latest first, each with sources.
Researcher Johann Rehberger reportedly drove Claude Code (Opus 5) in Auto Mode to code execution from a single summarize-this-website request, using a module-shadowing trick so an imported stdlib decoder runs an attacker payload, with a reported 60-80% success rate.
Varonis Threat Labs reportedly chained flaws in consumer Microsoft Copilot Personal so a crafted link with a hidden autorun parameter fired prompt execution on a single click, exfiltrated data from connected OAuth apps, and planted persistent memory rules that survived password resets and device re-enrollment; reportedly patched Aug 2026 with no evidence of abuse.
NVD reportedly published a CVSS 9.0 prompt-injection flaw in the widely-installed Context7 MCP documentation server, where unsanitized content served via its Custom AI Instructions feature is delivered into a connected coding agent's context and executed with the agent's own file, shell and network access, with no fix referenced at publication.
Anthropic and EPFL researchers reportedly showed self-propagating goals can spread between LLM agents through the editable system-prompt and state files that agent harnesses use to persist context, with some payloads surviving 20 transmission rounds in simulated multi-agent collaborations.
Microsoft Security reportedly catalogued an in-the-wild technique across dozens of companies where websites embed hidden prompt-injection payloads behind Ask-AI deep-links that, clicked in an authenticated assistant session, silently write a permanently-trust-this-vendor tag into the assistant's long-term memory.
Researcher Johann Rehberger reportedly showed an attacker with admin access to a LiteLLM AI gateway can weaponize the legitimate model-update endpoint and the callback system to reroute victim traffic, log backend provider API keys, and inject forged tool-calls into agent responses after inference, bypassing prompt-level defenses.
A paper reportedly presenting the first Write-Execute-Forget lifecycle benchmark for agent memory poisoning across many harness, memory-backend and model configurations, reporting that malicious memories persist in most cases and the full write-to-execute chain succeeds about half the time, with repair effectiveness varying widely by backend.
Zenity Labs disclosed that two unvalidated URL parameters in ChatGPT's Workspace Agent Builder let a single crafted link, opened by a logged-in enterprise user, silently create, authorize and deploy an autonomous agent under the victim's identity and connectors — bypassing approval prompts and reportedly polling an attacker inbox for commands every five minutes. Reported to OpenAI 4 Jun 2026 and fixed 8 Jun 2026; no in-the-wild exploitation reported.
Manifold Security reported that Microsoft's official Azure DevOps MCP server returns instructions planted in a hidden HTML comment inside a pull-request description — invisible in the web UI but returned verbatim by the API — so a victim's AI review agent, acting under the victim's credentials, follows the hidden orders and reaches data the attacker could not access directly.
Researchers reportedly showed hidden text in a web page could make AWS's Kiro agentic IDE rewrite execution-sensitive config it controls (mcp.json, tasks.json) that auto-loads on folder open, turning a summarize-this-page request into zero-click code execution (reportedly patched in v0.11.130 / 0.11.x; the primary Intezer and AWS sources publish no CVSS).
Pillar Security reportedly disclosed eight sandbox-escape vulnerabilities across four AI coding agents (Cursor, OpenAI Codex CLI, Google Gemini CLI, Google Antigravity) over four days, finding that in nearly every case the agent did not break the sandbox directly but wrote a file that a trusted component outside the sandbox later ran, loaded or scanned.
A red-team study shows adversaries can hide prompt-injection payloads inside network-log fields (usernames, URLs, user-agents) that fire when a SOC analyst asks an LLM to triage the logs — reportedly reaching up to 88.2% success at concealing malicious activity or exfiltrating data, turning the audit trail itself into the injection channel.
+ 43 more via the mapped risk pages above.
Browse all real-world cases →Practise it — interactive scenarios
An 'Ask AI' button quietly plants a permanent 'trusted source' rule in your assistant's memory
A coding agent asks to write ./notes.txt — the file it actually overwrites is your SSH keys
A support email hides instructions — and the assistant obeys them
A poisoned issue makes the agent lie to the human who approves its actions
Compromise the pipeline that builds agents, and every new worker is born malicious
An auto-approving coding agent reads a poisoned page — and executes code it never should have
A fake Sentry error report hijacks a developer's coding agent into running a shell command
The forensic record is itself the attack surface — an agent's log is poisoned, then quietly rewritten
A newsletter the user asked to summarise quietly writes a false 'fact' into the agent's long-term memory — and it detonates weeks later
A planted 'standing goal' copies itself agent-to-agent through the team's shared config files
A shopping page tells the agent to do something the user never asked for
One click provisions an attacker-configured agent inside your own workspace
An attacker plants prompt injection in the audit trail — so the LLM that hunts them erases the evidence
A single poisoned document plants a standing instruction that survives every reset
Encoded public text is laundered across an agent handoff into an on-chain transfer
A screenshot that's harmless at full size becomes an order once the system shrinks it
A forged peer registers on the agent directory — and the planner enlists it
The eval gate that was supposed to catch the agent is itself the thing being attacked
A poisoned web page hijacks a research agent — and the planner acts on its behalf
A GUI agent clicks 'Continue' — but the screen moved, and it lands on 'Send'
An inbox summary quietly ships a secret to an attacker's server
Controls & guardrails that address this
295 proposedGuardrails across the risks mapped to MANAGE 2.4, grouped by control function. Filter by control category below.
Design the system prompt architecture with privilege separation and trust tier definitions at design stage.
Implement input sanitisation and injection detection filters covering known injection patterns and privilege escalation attempts.
Wrap all untrusted content in random delimiters and datamarking; instruct the model never to execute instructions inside the marked region. Gate release on injection eval results.
source: Microsoft 'Spotlighting' technique (Hines et al. 2024); OWASP Top 10 for LLM Apps LLM01:2025 Prompt Injection (segregate external content)Benchmark the classifier on a labelled injection corpus and tune the decision threshold. Sign off the operating point before deployment.
source: MITRE ATLAS AML.M0015 (Adversarial Input Detection); OWASP Top 10 for LLM Apps LLM01:2025 Prompt Injection; NIST AI RMF MEASURE 2.7Before inference, render a preview of the exact image (and dimensions) the model will receive after preprocessing, and either avoid silent downscaling or constrain ingest dimensions — so an attacker cannot hide a payload that only becomes legible after resampling. Closes the inspected-vs-delivered gap that text-based injection filters miss.
source: Case study: anamorpher-image-scaling-injection (Trail of Bits — Morozova & Hussain, 21 Aug 2025)Enumerate every tool/MCP response channel that can carry untrusted content and verify the prompt-injection guardrail (delimiting/spotlighting + taint tag) is applied to each — treating coverage as an invariant, with a test that fails deployment if any channel is unguarded. Prevents the failure mode where a guardrail is present on some tools (pipeline/wiki) but missing on another (PR descriptions) that then becomes the unguarded ingress.
source: Case study: azure-devops-mcp-confused-deputy (Manifold Security — Francisco Rosales, 21 Jul 2026)Track provenance at field granularity: taint every value derived from attacker-reachable content (sender/origin fields, resource or element identifiers, tool-call/response structures) and enforce a policy that a tainted value may never be resolved as trusted metadata, an authorisation origin, or a fabricated tool-execution history. Closes the trusted-data vs untrusted-data gap ADI exploits — which instruction-focused defenses (spotlighting, injection classifiers, task-alignment output guardrails) miss by construction because the payload carries no instruction and preserves the user's task.
source: Case study: agent-data-injection-adi (Choi et al., SNU/UIUC/Largosoft — arXiv:2607.05120, 06 Jul 2026)Select or fine-tune the foundation model for a trained instruction-hierarchy prior so system-prompt directives intrinsically outrank user- and tool-originated instructions, and gate release on role-precedence override evals quantifying the residual (behavioural, non-enforced) flip rate.
source: Interactive-control reconciliation: ctrl-instruction-hierarchy (partial coverage)Training the model to treat the app's standing instructions as more authoritative than anything a user or document says.
Giving the agent only the keys it needs for the current task, not a master key to everything.
Clearly fencing off outside text — 'everything between these marks is just data, not instructions' — so the model is less likely to obey it.
Cleaning documents as they enter the library — stripping hidden text and active instructions — and only ingesting from trusted places.
Controlling where the AI can send data, so secrets can't be quietly shipped to a stranger's address or website.
Pausing to ask a person before doing anything big or hard to undo — sending money, deleting data, emailing customers.
Being careful about what gets saved to long-term memory, labelling where it came from, and letting users see and delete their memories.
Giving each AI worker its own limited permissions and clearly labelling messages between them as 'untrusted until checked'.
Give every AI agent a verifiable ID badge, keep a guest list of which agents are allowed on the team, and check the badge on every message — so an impostor or an uninvited agent can't be trusted.
Conduct a prompt injection threat assessment at design stage covering all input vectors (user, tool, external data).
Penetration test all prompt injection pathways in the system. Prioritise external tool and document ingestion channels.
Build the versioned injection corpus into CI/CD as a pre-release gate. Baseline attack success and sign off the release threshold.
source: NIST AI RMF MANAGE 2.2 / MEASURE 2.7; MITRE ATLAS AML.M0019 (Red Teaming); OWASP Top 10 for LLM Apps LLM01:2025 (adversarial testing)Log the exact post-truncation context the model ingested, including retrieved and tool-returned content rather than only user input, with redaction applied at read time, so indirect injection via that content is forensically visible.
source: Interactive-control reconciliation: ctrl-logging (partial coverage)A screen that reads incoming messages and blocks obvious attacks or banned topics before the model sees them.
Live dashboards and alarms that notice unusual behaviour — spikes in errors, weird actions, sudden data access.
Keeping a label on every document saying where it came from, so you can tell trusted company docs from random web text.
Recording everything — questions, documents fetched, actions taken — so you can investigate when something goes wrong.
Watching for strange new memories — like instructions that suddenly appear — and holding them aside until checked.
Automatic stop-switches when AIs get stuck in loops, burn too much money, or start disagreeing with each other.
Conduct comprehensive prompt injection red team exercises (direct, indirect, multi-turn) before deployment.
Classify content sources into trust tiers at design; place privileged tools behind a tier requiring user-originated intent or human approval. Sign off the trust-tier map before build.
source: Google DeepMind CaMeL (2025); OWASP Agentic AI Threats & Mitigations (tool misuse / compromise); NIST SP 800-53 AC-6 Least PrivilegeRe-run injection evals on every template change and periodically against new attack techniques. Manage the spotlighting wrapper under change control.
source: Microsoft 'Spotlighting' technique (Hines et al. 2024); OWASP Top 10 for LLM Apps LLM01:2025 Prompt Injection (segregate external content)