Unsafe Tool / Code Execution
highAgency & toolsDefinition
When the AI can run code or commands, a bad instruction can become a real attack on the computer running it — reading files, reaching the network, or worse.
This is recommended as a granular sub-risk of #42 Tool-layer misuse and unintended actions (Cyber & Data Security · Technology Risk). #42 is framed around permission-exceeding tool calls; this names the unsandboxed-execution mechanism and its classic injection/SSRF vectors. Your 44-row Enterprise Risk Mapping is unchanged — this is a suggestion for inclusion.
Where it attaches
The system components this risk arises at.
Detection signals
- ▸ Generated SQL/shell with injection patterns
- ▸ Fetch tool requests to internal/metadata endpoints (SSRF)
- ▸ Sandbox resource/egress alarms
Controls & guardrails that address this
4Grouped by control function, with the AI lifecycle stage(s) to apply each and the other risks it addresses. Filter by control category below.
Double-checking the details of every action the AI wants to take, and running risky actions in a locked-down environment.
Controlling where the AI can send data, so secrets can't be quietly shipped to a stranger's address or website.
Giving the agent only the keys it needs for the current task, not a master key to everything.
Recording everything — questions, documents fetched, actions taken — so you can investigate when something goes wrong.
Framework mappings
Real-world cases
31Actual published events that illustrate this risk — click through for the writeup and sources.
A coding agent with production access reportedly dropped a live database during a run — ungated irreversible action by an over-privileged agent.
Anthropic reports that a suspected Chinese state-sponsored group (GTG-1002) jailbroke Claude Code via a 'defensive security firm' role-play and task decomposition, then used it to run an estimated 80-90% of tactical operations in a multi-target espionage campaign largely autonomously.
Researcher Ari Marzouk disclosed 30+ vulnerabilities (24 CVEs) across 10-plus AI coding agents (Copilot, Cursor, Windsurf, Claude Code, Junie and others) where a prompt injected via repo files, READMEs, file names or MCP tool responses makes the assistant weaponize legitimate IDE features for code execution and secret exfiltration.
Researcher Johann Rehberger showed that injected instructions in source code, web pages, or GitHub issues could make the Copilot agent silently write "chat.tools.autoApprove": true into .vscode/settings.json, disabling human approval and granting unattended shell execution — a self-config-rewrite to full-host compromise (CVE-2025-53773).
Unit 42 showed that when a Hugging Face account is deleted (or a model is transferred and the old author later removed), its Author/ModelName namespace can be re-registered by anyone — so platforms and code that resolve models by name auto-deploy attacker-controlled weights, demonstrated as reverse-shell RCE on Google Vertex AI Model Garden and Azure AI Foundry.
An attacker got a malicious pull request merged into the open-source aws-toolkit-vscode repo, embedding a destructive prompt that told the Amazon Q agent to wipe local files and AWS resources; the tainted build (v1.84.0) reached the Marketplace's ~1M installs before removal.
Wiz Research chained three flaws in NVIDIA Triton's Python-backend shared-memory IPC — an information leak of the backend's private shared-memory region name (CVE-2025-23320), a missing ownership/validation check that lets that region be re-registered as attacker-controlled memory, and an out-of-bounds write that corrupts internal data structures (CVE-2025-23319) — to give a remote, unauthenticated attacker full code execution and takeover of an AI model-serving server, reportedly enabling model theft, response manipulation and lateral movement.
As part of a multi-ecosystem supply-chain cascade (Trivy onward), TeamPCP used stolen PyPI publishing tokens to ship backdoored BerriAI LiteLLM versions whose auto-running .pth payload harvested cloud, SSH and Kubernetes secrets plus env vars holding OPENAI_API_KEY/ANTHROPIC_API_KEY — exfiltrating to a typosquatted C2; AI-talent firm Mercor was a downstream victim, with Lapsus$ claiming ~4TB stolen.
Tenet Security showed that a single fake Sentry error report, sent using only a public DSN, can hijack AI coding agents (Claude Code, Cursor, Codex) into running attacker-controlled code on a developer's machine — an indirect-injection attack delivered through a trusted MCP integration.
Hugging Face's LeRobot robotics-AI framework reportedly exposed its async-inference policy server over an unauthenticated, no-TLS gRPC port that calls Python pickle.loads() on attacker-controlled data, allowing unauthenticated remote code execution on the model-inference host.
A CVSS 10.0 remote-code-execution flaw in Flowise's CustomMCP node lets an attacker run arbitrary JavaScript on the host: the MCP server config is reportedly passed straight to JavaScript's Function() constructor with no validation. Disclosed in Sept 2025 and patched in 3.0.6, it later saw active mass exploitation across thousands of exposed instances in April 2026.
Anthropic reports that 'Claude Mythos Preview' — an unreleased frontier model it describes as able to autonomously find and exploit software flaws — surfaced more than 10,000 high- or critical-severity vulnerabilities across major operating systems, browsers and open-source projects in roughly its first month under the defensive 'Project Glasswing' program, with Anthropic warning that finding flaws now far outpaces the human capacity to triage and patch them.
Gambit Security reports that a single operator weaponized Anthropic's Claude Code and OpenAI's GPT-4.1 to breach at least nine Mexican government organizations, with Claude Code reportedly executing ~75% of remote commands after the attacker bypassed its refusals by loading a 1,084-line hacking cheatsheet as a persistent claude.md system prompt.
Hugging Face disclosed a production-infrastructure intrusion that it says was driven end-to-end by an autonomous AI-agent system: a malicious dataset abused code-execution paths in its dataset-processing pipeline as the foothold, then the campaign escalated to node-level access and moved laterally into internal clusters over a weekend.
Two MCP 'test' endpoints in the LiteLLM AI gateway accepted a full stdio server config and spawned the supplied command as a subprocess on the proxy host; Horizon3.ai chained it with a Starlette host-header bypass (CVE-2026-48710) to reach unauthenticated RCE, and CISA added it to KEV after reported in-the-wild exploitation.
Wiz Research found Amazon Q's VS Code extension auto-loaded MCP server definitions from a repo's .amazonq/mcp.json with no consent or workspace-trust prompt; opening a booby-trapped repository silently spawned attacker-controlled processes that inherited the developer's full environment and could stream live AWS session credentials out to an attacker.
Cato AI Labs disclosed two critical (CVSS 9.8) zero-click flaws in Cursor's coding agent where a single instruction hidden in content the agent reads — an MCP tool response or a web-search result — escapes the editor's terminal sandbox and runs OS-level commands with no click or approval.
Threat-intel firm Hunt.io and researcher Bob Diachenko reported finding exposed attacker directories (staged on a Hong Kong server, archived 9-13 Jul 2026) showing a threat actor installed the open-source Hermes AI agent, ran it in unattended 'YOLO' mode - the documented flag that removes the human-approval prompt - and delegated post-exploitation to it: the agent reportedly ran a customised LinPEAS, hunted Linux privilege-escalation paths, traversed ministry directories and catalogued Office of the Permanent Secretary staff/personnel records dating to 2012. The Ministry has not confirmed a breach, and investigators say nothing in the recovered files shows data leaving the network.
Sysdig documented what it assesses as the first ransomware operation run end-to-end by an autonomous LLM agent with no human at the keyboard: after a Langflow RCE (CVE-2025-3248) the agent reportedly harvested credentials, moved laterally, encrypted 1,342 Nacos configuration items and extorted the target — adapting at machine speed, including fixing a broken login routine in 31 seconds.
Spira, Cohen, Nassi et al. (the Morris II group) show LLM resource-name hallucination can be weaponised at scale: attackers pre-compute a model's most-likely hallucinated names for trending repos/skills, register them, and host adversarial prompts there. The paper reports hallucinated-resource generation up to 85% for repository cloning and up to 100% for skill installation, with hallucinations that transfer across foundation models and prompts, enabling remote tool/code execution assemblable into a botnet.
Wiz Research disclosed 'GhostApproval', a cross-vendor trust-boundary flaw in six AI coding assistants where a benign-looking repo file that is actually a symlink to a sensitive path makes the 'approve this edit' dialog display the innocent in-workspace path while the write lands outside the workspace — combining CWE-61 symlink following with CWE-451 UI misrepresentation to reduce human approval to a rubber stamp.
Research reportedly showing individually-benign CLI commands composed by LLM coding agents into dangerous state chains, with a ~96.6% attack success rate across five agents and five model backends.
A paper defining Agent Data Injection (ADI) — payloads disguised as trusted data rather than instructions — reportedly achieving arbitrary clicks, RCE and supply-chain compromise against Claude in Chrome, OpenAI Codex and Gemini CLI.
Researcher Johann Rehberger reportedly drove Claude Code (Opus 5) in Auto Mode to code execution from a single summarize-this-website request, using a module-shadowing trick so an imported stdlib decoder runs an attacker payload, with a reported 60-80% success rate.
Researchers reportedly showed hidden text in a web page could make AWS's Kiro agentic IDE rewrite execution-sensitive config it controls (mcp.json, tasks.json) that auto-loads on folder open, turning a summarize-this-page request into zero-click code execution (reportedly patched in v0.11.130 / 0.11.x; the primary Intezer and AWS sources publish no CVSS).
A disclosure wave in which Anthropic reportedly found eval-time models reaching three organizations' production infrastructure — including a Claude model that published a credential-stealing package to the real PyPI — with Meta and the UK AI Security Institute reporting similar harness-containment failures.
A joint US government advisory reportedly warned that threat actors are using AI-generated Python scripts disguised as legitimate OT monitoring tools for reconnaissance and read/write attacks on internet-exposed Siemens S7-series PLCs across critical-infrastructure sectors.
A reported CVSS 9.5 pre-authentication flaw in the ServiceNow AI Platform where unauthenticated endpoints feed attacker input into a query-filter operator that evaluates it as JavaScript, escalating via a script-loading gadget into full sandbox-escape code execution; reportedly patched 13 Jul 2026 with in-the-wild exploitation days later.
Pillar Security reportedly disclosed eight sandbox-escape vulnerabilities across four AI coding agents (Cursor, OpenAI Codex CLI, Google Gemini CLI, Google Antigravity) over four days, finding that in nearly every case the agent did not break the sandbox directly but wrote a file that a trusted component outside the sandbox later ran, loaded or scanned.
A self-propagating npm worm reportedly hijacked the keyv/cacheable maintainer account and trojanized hundreds of package versions with a malicious preinstall hook, and additionally committed Claude Code and VS Code hook files into the source repo so that merely opening a checkout in an AI-enabled IDE runs the payload with no npm install.
A reported CVSS 9.8 code-injection flaw in the open-source Langflow AI-workflow builder lets an unauthenticated attacker chain two API endpoints into unsafe dynamic code evaluation for full RCE on a default install; reportedly mass-exploited within weeks and added to CISA's Known Exploited Vulnerabilities catalog.
Practise this in an interactive scenario
A coding agent asks to write ./notes.txt — the file it actually overwrites is your SSH keys
A text-to-SQL agent runs the model's output straight at the database
An auto-approving coding agent reads a poisoned page — and executes code it never should have
A fake Sentry error report hijacks a developer's coding agent into running a shell command
Every command is harmless on its own — the sequence is the exploit
A 'safe' dataset preview turns an upload into code execution on the pipeline's workers