Extract LLM System Prompt
How AML.T0056 Extract LLM System Prompt shows up in practice: the mapped risk classes in this atlas, the documented incidents that prove it's real, and the scenarios and controls to learn and defend against it.
Mapped risks
Risk classes in this atlas that map to AML.T0056 — click through for the full definition, attack surface and controls.
Real-world cases
5Documented incidents, disclosed vulnerabilities and research that illustrate AML.T0056 — latest first, each with sources.
Anthropic reports that 'Claude Mythos Preview' — an unreleased frontier model it describes as able to autonomously find and exploit software flaws — surfaced more than 10,000 high- or critical-severity vulnerabilities across major operating systems, browsers and open-source projects in roughly its first month under the defensive 'Project Glasswing' program, with Anthropic warning that finding flaws now far outpaces the human capacity to triage and patch them.
Crowd-sourced GitHub repos systematically extract and publish system prompts AND JSON tool/function schemas from deployed AI agents (Cursor, Windsurf, Claude Code, Devin, Copilot), one hitting ~140k stars.
Wallarm reported jailbreaking DeepSeek's chatbot to extract its full system prompt verbatim using a 'bias-based' technique; DeepSeek deployed a fix.
A CCS'24 paper that optimizes adversarial queries to reconstruct hidden system prompts, exactly recovering them for 68% of 50 real deployed Poe LLM apps.
Users extracted Bing Chat's hidden system instructions and internal codename 'Sydney' via direct prompt injection shortly after launch.
Controls & guardrails that address this
5Guardrails across the risks mapped to AML.T0056, grouped by control function. Filter by control category below.
Training the model to treat the app's standing instructions as more authoritative than anything a user or document says.
Giving the agent only the keys it needs for the current task, not a master key to everything.
A screen that reads incoming messages and blocks obvious attacks or banned topics before the model sees them.
Live dashboards and alarms that notice unusual behaviour — spikes in errors, weird actions, sudden data access.
The organisational habits around the AI: assessing risks before launch, actively trying to break it, and having a plan for when something goes wrong.