Evade AI Model
How AML.T0015 Evade AI Model shows up in practice: the mapped risk classes in this atlas, the documented incidents that prove it's real, and the scenarios and controls to learn and defend against it.
Mapped risks
Risk classes in this atlas that map to AML.T0015 — click through for the full definition, attack surface and controls.
The flight recorder and the alarms can themselves be attacked. If logs can be erased or rewritten, fake entries slipped in, or the monitors quietly evaded, the one record you'd rely on to notice and investigate an incident is no longer trustworthy.
Even if the model itself is genuine, the machinery running it can be tweaked at the moment of answering — nudging its 'thoughts' or biasing word choice — in ways that leave no trace in the model file.
Real-world cases
7Documented incidents, disclosed vulnerabilities and research that illustrate AML.T0015 — latest first, each with sources.
An investigation reportedly described a state-linked campaign publishing content phrased as chatbot questions (generative engine optimization) so that ChatGPT and Perplexity retrieve and cite it when answering neutral questions about Gaza and the IDF.
A paper reportedly showing that the encrypted reasoning envelopes returned by Anthropic/OpenAI/Google APIs are interchangeable across a provider's own models, so replaying one into a weaker sibling recovers the hidden chain-of-thought in plaintext, with PII and credentials extracted from scraped blocks.
A red-team study shows adversaries can hide prompt-injection payloads inside network-log fields (usernames, URLs, user-agents) that fire when a SOC analyst asks an LLM to triage the logs — reportedly reaching up to 88.2% success at concealing malicious activity or exfiltrating data, turning the audit trail itself into the injection channel.
Johann Rehberger showed a time-of-check/time-of-use race in GUI/computer-use agents: the screen can change after the agent captures its screenshot but before its click lands, so a benign-looking 'Continue' button silently resolves to an Outlook 'Send'. Anthropic tracked the issue and Cowork now revalidates pixels before acting.
Gambit Security reports that a single operator weaponized Anthropic's Claude Code and OpenAI's GPT-4.1 to breach at least nine Mexican government organizations, with Claude Code reportedly executing ~75% of remote commands after the attacker bypassed its refusals by loading a 1,084-line hacking cheatsheet as a persistent claude.md system prompt.
Heretic automates 'abliteration' — removing an open model's safety refusals by orthogonalizing the refusal direction out of its weights, with an Optuna search that preserves capability — and has produced 4000+ uncensored models on Hugging Face.
Model behaviour can be steered by adding directions to activations at inference — usable for control, or for covert manipulation.
Practise it — interactive scenarios
Subtract the refusal direction during generation — safety off, weights untouched
A compromised serving stack edits the model's activations — the weight hash never changes
The forensic record is itself the attack surface — an agent's log is poisoned, then quietly rewritten
An attacker plants prompt injection in the audit trail — so the LLM that hunts them erases the evidence
The eval gate that was supposed to catch the agent is itself the thing being attacked
Controls & guardrails that address this
163 proposedGuardrails across the risks mapped to AML.T0015, grouped by control function. Filter by control category below.
Implement adversarial example detection at the inference boundary. Block or flag inputs matching known attack patterns.
Sign and hash-register every model and adapter with a provenance manifest at onboarding. Refuse registry admission for unsigned artifacts.
source: MITRE ATLAS AML.M0013 (Code Signing), AML.M0014 (Verify ML Artifacts); NIST SP 800-53 SI-7 Software, Firmware, and Information Integrity; CSA MAESTRO supply-chain layerSample classifier verdicts and breaker trips on a cadence; retune thresholds and update signatures for confirmed misses.
source: OWASP Top 10 for LLM Apps LLM01:2025 Prompt Injection; MITRE ATLAS AML.M0015 (Adversarial Input Detection); NIST SP 800-53 SI-4 System Monitoring, SC-5Making sure the machinery running the model — and the template used to stamp out new agents — is the real, unmodified version, and that one user's data can't leak into another's through shared shortcuts.
Recording everything — questions, documents fetched, actions taken — so you can investigate when something goes wrong.
Live dashboards and alarms that notice unusual behaviour — spikes in errors, weird actions, sudden data access.
Conduct an adversarial manipulation threat assessment at design stage. Identify attack vectors and rate residual risk.
Run adaptive multi-turn jailbreak fuzzing against every release candidate. Gate release on attack-success rate within threshold and re-test each fixed bypass.
source: OWASP Top 10 for LLM Apps LLM01:2025 Prompt Injection; MITRE ATLAS AML.M0019 (Red Teaming); NIST AI RMF MEASURE 2.7Assemble the golden probe set and baseline pass rates before first release. Obtain risk-owner approval of coverage and thresholds.
source: NIST AI RMF MEASURE 2.7 and MANAGE 4.1; MITRE ATLAS AML.M0015 (Adversarial Input Detection / monitoring); NIST SP 800-53 SI-4, CM-3 Configuration Change ControlOn the AI provider/platform side, detect sustained abuse independent of any single refusal: per-principal analytics on remote-command-execution volume and external-target breadth, anti-forensic tradecraft, and bulk-data API processing — with rate-limit / session kill-switch on confirmed abuse. Make refusal stateful so a refused objective cannot be re-entered as a persisted auto-loaded context file (e.g. claude.md), and treat writes into auto-loaded model-context files as security-relevant. Closes the gap that per-turn refusal leaves when the operator is the adversary.
source: Case study: gambit-mexico-gov-ai-breach (Gambit Security / Eyal Sela technical report; campaign began 27 Dec 2025, reported through mid-Feb 2026)For a hosted agentic tool with dual-use capability, detect misuse from account behaviour across sessions — not just per-turn refusal. Correlate signals such as offensive tool/command patterns (network scanning, credential/hash attacks), real external victim identifiers (IPs, hostnames, credentials) appearing in a coding/agent context, and repeated refuse-then-reframe loops, and treat self-asserted context ('this is a test environment') as unverified rather than as authorization. Wire the detection to rate-limits, account suspension, live-session revocation, and provider incident response. Closes the Aur0ra/Cursor vector where an intent-laundering jailbreak defeated a stateless refusal across 28 sessions for six weeks undetected; complements refusal training, jailbreak evals, and instruction-hierarchy hardening rather than relying on them.
source: Case study: aur0ra-cursor-ai-ransomwareRegularly testing the AI against a set of known-good and known-bad examples, and re-testing whenever anything changes.
The organisational habits around the AI: assessing risks before launch, actively trying to break it, and having a plan for when something goes wrong.
Conduct adversarial robustness testing (white-box, black-box, transfer attacks) before deployment.
Penetration test the model inference layer to identify specific adversarial input vulnerabilities.
Score every prompt and response with an inline safety classifier; trip a circuit breaker on sessions with sustained anomalous scores. Keep thresholds under change control.
source: OWASP Top 10 for LLM Apps LLM01:2025 Prompt Injection; MITRE ATLAS AML.M0015 (Adversarial Input Detection); NIST SP 800-53 SI-4 System Monitoring, SC-5Re-run the jailbreak fuzzing harness on a recurring cadence with newly observed attack techniques added. Escalate threshold breaches for remediation.
source: OWASP Top 10 for LLM Apps LLM01:2025 Prompt Injection; MITRE ATLAS AML.M0019 (Red Teaming); NIST AI RMF MEASURE 2.7Require measured-boot/runtime attestation of the inference serving binary and partition KV/prefix caches per tenant, closing decode-time serving-layer tampering and co-tenancy timing side channels that artifact weight-hashing cannot detect.
source: Interactive-control reconciliation: ctrl-stack-attestation (partial coverage)