Prompt injection

Users extracted Bing Chat's hidden system instructions and internal codename 'Sydney' via direct prompt injection shortly after launch.

An attacker got a malicious pull request merged into the open-source aws-toolkit-vscode repo, embedding a destructive prompt that told the Amazon Q agent to wipe local files and AWS resources; the tainted build (v1.84.0) reached the Marketplace's ~1M installs before removal.

Researchers report that adaptive attackers bypass 12 recent jailbreak and prompt-injection defenses with attack success rates above 90% for most, despite those defenses having originally reported near-zero success rates.

A single malicious link reportedly turned Copilot Enterprise Search's URL query parameter into an executable prompt, exfiltrating emails, MFA codes and files via a Bing image-search side channel.

An X user escalated Grok's on-chain wallet via a Bankr Club NFT, then sent a Morse-code instruction Grok auto-decoded and relayed to the autonomous agent Bankrbot — moving ~3B tokens (reportedly ~$150K-$200K) with no secondary verification.

Cato AI Labs disclosed two critical (CVSS 9.8) zero-click flaws in Cursor's coding agent where a single instruction hidden in content the agent reads — an MCP tool response or a web-search result — escapes the editor's terminal sandbox and runs OS-level commands with no click or approval.

A red-team study shows adversaries can hide prompt-injection payloads inside network-log fields (usernames, URLs, user-agents) that fire when a SOC analyst asks an LLM to triage the logs — reportedly reaching up to 88.2% success at concealing malicious activity or exfiltrating data, turning the audit trail itself into the injection channel.

Zenity Labs disclosed that two unvalidated URL parameters in ChatGPT's Workspace Agent Builder let a single crafted link, opened by a logged-in enterprise user, silently create, authorize and deploy an autonomous agent under the victim's identity and connectors — bypassing approval prompts and reportedly polling an attacker inbox for commands every five minutes. Reported to OpenAI 4 Jun 2026 and fixed 8 Jun 2026; no in-the-wild exploitation reported.

OpenAI detailed GPT-Red, an internal self-play red-teaming model that reportedly beat human red-teamers 84% to 13% on prompt-injection tasks and autonomously surfaced a novel 'Fake Chain-of-Thought' attack — planting a spoofed, already-verified reasoning step in a model's trace to smuggle attacker instructions past its checks.

Researcher Johann Rehberger reportedly drove Claude Code (Opus 5) in Auto Mode to code execution from a single summarize-this-website request, using a module-shadowing trick so an imported stdlib decoder runs an attacker payload, with a reported 60-80% success rate.

Varonis Threat Labs reportedly chained flaws in consumer Microsoft Copilot Personal so a crafted link with a hidden autorun parameter fired prompt execution on a single click, exfiltrated data from connected OAuth apps, and planted persistent memory rules that survived password resets and device re-enrollment; reportedly patched Aug 2026 with no evidence of abuse.

NVD reportedly published a CVSS 9.0 prompt-injection flaw in the widely-installed Context7 MCP documentation server, where unsanitized content served via its Custom AI Instructions feature is delivered into a connected coding agent's context and executed with the agent's own file, shell and network access, with no fix referenced at publication.

Roleplay framings ('my late grandma used to read me…') coaxed chatbots past safety training into producing restricted content.

Optimised gibberish suffixes that transfer across models to reliably elicit refused content — automated, transferable jailbreaks.

Filling a long context with many faux-compliant dialogue examples erodes a model's refusals — an attack that scales with context length.

Anthropic reports that a suspected Chinese state-sponsored group (GTG-1002) jailbroke Claude Code via a 'defensive security firm' role-play and task decomposition, then used it to run an estimated 80-90% of tactical operations in a multi-target espionage campaign largely autonomously.

Wallarm reported jailbreaking DeepSeek's chatbot to extract its full system prompt verbatim using a 'bias-based' technique; DeepSeek deployed a fix.

Matthew and Maria Raine sued OpenAI and CEO Sam Altman (San Francisco Superior Court, 26 Aug 2025) over the April 2025 suicide of their 16-year-old son Adam, alleging ChatGPT fostered psychological dependency, discouraged him from confiding in family, and supplied self-harm method detail — while he reportedly circumvented its safeguards for months by framing queries as fiction. OpenAI denies liability, saying it pointed him to crisis resources 100+ times and that he misused the product. (Allegations unproven; litigation ongoing.)

Rewriting a harmful request as a poem bypasses safety alignment across 25 frontier proprietary and open-weight LLMs: hand-crafted poems reached ~62% average attack-success (some providers >90%), and mechanically converting harmful prompts to verse raised success up to 18x over prose baselines.

Gambit Security reports that a single operator weaponized Anthropic's Claude Code and OpenAI's GPT-4.1 to breach at least nine Mexican government organizations, with Claude Code reportedly executing ~75% of remote commands after the attacker bypassed its refusals by loading a 1,084-line hacking cheatsheet as a persistent claude.md system prompt.

A Reuters exclusive reportedly documented a Russian-speaking ransomware group using the Cursor AI coding assistant as a hacking copilot, jailbreaking its guardrails with a simulation/test-environment framing to obtain vulnerability-identification, credential-theft and exploitation guidance against at least seven companies.

A crafted email's hidden instructions made M365 Copilot exfiltrate tenant data via an auto-rendered image URL — with no user click.

An academic paper showed instructions hidden in a webpage hijacking an LLM-integrated app reading it — coining 'indirect prompt injection'.

Researchers showed web-browsing AI agents following instructions embedded in attacker-controlled pages to leak data or take actions.

Indirect injection could write attacker instructions into ChatGPT's long-term memory, persisting across chats to exfiltrate data until OpenAI mitigated it.

Hidden instructions embedded in MCP tool descriptions hijacked agents (e.g. in Cursor) that merely listed the available tools.

Microsoft AI Red Team whitepaper enumerating agentic failure modes, including resource/service exhaustion from runaway loops and fan-out.

Researchers showed attacker text planted in a public Salesforce Web-to-Lead form is later read by the Agentforce agent during normal use and treated as instructions, exfiltrating CRM data to an attacker domain that had been on Salesforce's CSP allow-list but expired and was re-registered for about $5.

AppOmni showed ServiceNow Now Assist's default agent config lets a malicious ticket redirect a benign agent into enlisting a more powerful agent — performing record CRUD, admin-role assignment, and email exfiltration with the triggering user's privilege, despite built-in prompt-injection protection.

A single crafted email with hidden HTML instructions reportedly made OpenAI's Deep Research agent autonomously exfiltrate Gmail inbox data from OpenAI's own cloud — with no user click and, per Radware, no client-side or network evidence.

Researcher Ari Marzouk disclosed 30+ vulnerabilities (24 CVEs) across 10-plus AI coding agents (Copilot, Cursor, Windsurf, Claude Code, Junie and others) where a prompt injected via repo files, READMEs, file names or MCP tool responses makes the assistant weaponize legitimate IDE features for code execution and secret exfiltration.

Researcher Johann Rehberger showed that injected instructions in source code, web pages, or GitHub issues could make the Copilot agent silently write "chat.tools.autoApprove": true into .vscode/settings.json, disabling human approval and granting unattended shell execution — a self-config-rewrite to full-host compromise (CVE-2025-53773).

A red-team PoC forged an inflated A2A 'agent card' so the orchestrator's LLM-as-judge routing always selected the rogue agent, diverting every task through the attacker.

Unit 42 PoCs in which a malicious remote agent abuses default inter-agent trust to covertly inject extra instructions across a stateful A2A session, invisible to the human operator.

Cohen, Bitton & Nassi (arXiv Mar 2024; ACM CCS 2025) built 'Morris II', the first worm targeting GenAI ecosystems: an adversarial self-replicating prompt that, via RAG-based inference, triggers a zero-click chain of indirect injections forcing each agent to act maliciously and re-infect the next — demonstrated stealing data and spamming through email assistants on ChatGPT, Gemini and LLaVA.

Trail of Bits showed an image that looks benign at full resolution exposes a hidden prompt-injection payload once an AI pipeline downscales it, and used it against Gemini CLI to silently exfiltrate Google Calendar data through an auto-approved Zapier tool call.

A benchmark of LLM-agent susceptibility to tool poisoning via malicious tool metadata, built on 45 live MCP servers and 353 real tools; the authors report agents are rarely able to refuse and that more-capable models are often more vulnerable.

Tenet Security showed that a single fake Sentry error report, sent using only a public DSN, can hijack AI coding agents (Claude Code, Cursor, Codex) into running attacker-controlled code on a developer's machine — an indirect-injection attack delivered through a trusted MCP integration.

Attacker-controlled Markdown hidden in a public web page is reportedly rendered by ChatGPT's summarization feature as trusted assistant output — spoofed OpenAI alerts, phishing links, QR codes, and tracking pixels.

GMO Flatt Security's RyotaK showed that a single attacker-opened GitHub issue could indirect-prompt-inject Anthropic's claude-code-action CI agent — whose permission check reportedly trusted any "[bot]" actor — coaxing Claude to leak CI secrets and OIDC tokens, gain repository write access, and potentially poison the shared action that downstream repos pull via a floating tag.

ThreatLabz documented two deployed web campaigns that hid instructions in pages (off-screen CSS text and JSON-LD metadata) to steer AI browsing agents — a fake Python-docs site inducing a bogus $3 API-key payment and a DeBank impersonation site pushed as 'authoritative'; across 26 LLMs, 4 executed the fake payment and 2 endorsed the scam site.

Manifold Security reported that Microsoft's official Azure DevOps MCP server returns instructions planted in a hidden HTML comment inside a pull-request description — invisible in the web UI but returned verbatim by the API — so a victim's AI review agent, acting under the victim's credentials, follows the hidden orders and reaches data the attacker could not access directly.

An academic paper reports MemGhost, a one-shot indirect-injection attack in which a single ordinary email causes a memory-enabled personal agent to silently write an attacker's false fact into long-term memory, conceal the write from its reply, and rely on the fiction in later sessions.

Spira, Cohen, Nassi et al. (the Morris II group) show LLM resource-name hallucination can be weaponised at scale: attackers pre-compute a model's most-likely hallucinated names for trending repos/skills, register them, and host adversarial prompts there. The paper reports hallucinated-resource generation up to 85% for repository cloning and up to 100% for skill installation, with hallucinations that transfer across foundation models and prompts, enabling remote tool/code execution assemblable into a botnet.

A paper defining Agent Data Injection (ADI) — payloads disguised as trusted data rather than instructions — reportedly achieving arbitrary clicks, RCE and supply-chain compromise against Claude in Chrome, OpenAI Codex and Gemini CLI.

Researchers reportedly showed hidden text in a web page could make AWS's Kiro agentic IDE rewrite execution-sensitive config it controls (mcp.json, tasks.json) that auto-loads on folder open, turning a summarize-this-page request into zero-click code execution (reportedly patched in v0.11.130 / 0.11.x; the primary Intezer and AWS sources publish no CVSS).

Microsoft Security reportedly catalogued an in-the-wild technique across dozens of companies where websites embed hidden prompt-injection payloads behind Ask-AI deep-links that, clicked in an authenticated assistant session, silently write a permanently-trust-this-vendor tag into the assistant's long-term memory.

Pillar Security reportedly disclosed eight sandbox-escape vulnerabilities across four AI coding agents (Cursor, OpenAI Codex CLI, Google Gemini CLI, Google Antigravity) over four days, finding that in nearly every case the agent did not break the sandbox directly but wrote a file that a trusted component outside the sandbox later ran, loaded or scanned.

Srivastav & Zhang (REALM 2025) showed a role-based multi-agent framework that splits a harmful request into individually-benign sub-questions, answers each separately, then reassembles the fragments into prohibited content — reportedly exceeding 90% attack success across three models.