Vector and Embedding Weaknesses
How LLM08:2025 Vector and Embedding Weaknesses shows up in practice: the mapped risk classes in this atlas, the documented incidents that prove it's real, and the scenarios and controls to learn and defend against it.
Mapped risks
Risk classes in this atlas that map to LLM08:2025 — click through for the full definition, attack surface and controls.
Someone slips bad information into the documents the AI learns from or looks things up in — so it confidently repeats falsehoods or follows planted instructions.
Private information escapes — the AI reveals secrets in its answer, or an attacker tricks it into emailing or posting your data somewhere they control.
Real-world cases
38Documented incidents, disclosed vulnerabilities and research that illustrate LLM08:2025 — latest first, each with sources.
Varonis Threat Labs reportedly chained flaws in consumer Microsoft Copilot Personal so a crafted link with a hidden autorun parameter fired prompt execution on a single click, exfiltrated data from connected OAuth apps, and planted persistent memory rules that survived password resets and device re-enrollment; reportedly patched Aug 2026 with no evidence of abuse.
An investigation reportedly described a state-linked campaign publishing content phrased as chatbot questions (generative engine optimization) so that ChatGPT and Perplexity retrieve and cite it when answering neutral questions about Gaza and the IDF.
A paper reportedly showing that the encrypted reasoning envelopes returned by Anthropic/OpenAI/Google APIs are interchangeable across a provider's own models, so replaying one into a weaker sibling recovers the hidden chain-of-thought in plaintext, with PII and credentials extracted from scraped blocks.
Researcher Johann Rehberger reportedly showed an attacker with admin access to a LiteLLM AI gateway can weaponize the legitimate model-update endpoint and the callback system to reroute victim traffic, log backend provider API keys, and inject forged tool-calls into agent responses after inference, bypassing prompt-level defenses.
Manifold Security reported that Microsoft's official Azure DevOps MCP server returns instructions planted in a hidden HTML comment inside a pull-request description — invisible in the web UI but returned verbatim by the API — so a victim's AI review agent, acting under the victim's credentials, follows the hidden orders and reaches data the attacker could not access directly.
Hugging Face disclosed a production-infrastructure intrusion that it says was driven end-to-end by an autonomous AI-agent system: a malicious dataset abused code-execution paths in its dataset-processing pipeline as the foothold, then the campaign escalated to node-level access and moved laterally into internal clusters over a weekend.
A red-team study shows adversaries can hide prompt-injection payloads inside network-log fields (usernames, URLs, user-agents) that fire when a SOC analyst asks an LLM to triage the logs — reportedly reaching up to 88.2% success at concealing malicious activity or exfiltrating data, turning the audit trail itself into the injection channel.
A reported CVSS 9.5 pre-authentication flaw in the ServiceNow AI Platform where unauthenticated endpoints feed attacker input into a query-filter operator that evaluates it as JavaScript, escalating via a script-loading gadget into full sandbox-escape code execution; reportedly patched 13 Jul 2026 with in-the-wild exploitation days later.
A security researcher (Cereblab) captured xAI's Grok Build CLI silently uploading complete local Git repositories — untracked working files, full commit history, and unredacted secrets — to a Google Cloud Storage bucket, reportedly roughly 27,800x more data than the coding task needed, with the user-facing privacy toggle having no effect on the uploads.
mem0's openmemory/api registered routers with no auth: an unauthenticated attacker could read/write/delete any user's stored memories (or globally pause memory for DoS), while a companion flaw exposed stored LLM API keys in plaintext and enabled SSRF to cloud metadata endpoints.
An academic paper reports MemGhost, a one-shot indirect-injection attack in which a single ordinary email causes a memory-enabled personal agent to silently write an attacker's false fact into long-term memory, conceal the write from its reply, and rely on the fiction in later sessions.
Wiz Research found Amazon Q's VS Code extension auto-loaded MCP server definitions from a repo's .amazonq/mcp.json with no consent or workspace-trust prompt; opening a booby-trapped repository silently spawned attacker-controlled processes that inherited the developer's full environment and could stream live AWS session credentials out to an attacker.
+ 26 more via the mapped risk pages above.
Browse all real-world cases →Practise it — interactive scenarios
An attacker edits the wiki; the assistant cites the lie back to everyone
A support email hides instructions — and the assistant obeys them
A speed optimisation becomes a cross-tenant listening device
An attacker crafts a gibberish passage whose embedding sits near thousands of questions — so it's retrieved everywhere
Two doors to the same secret: reconstruct the model through its API, or just walk off with the weight file
The safety guard is itself a trained model — and someone poisoned its lessons
The forensic record is itself the attack surface — an agent's log is poisoned, then quietly rewritten
A 'safe' dataset preview turns an upload into code execution on the pipeline's workers
A screenshot that's harmless at full size becomes an order once the system shrinks it
An attacker captures the agent's bearer token — and inherits its authority
A forged peer registers on the agent directory — and the planner enlists it
An inbox summary quietly ships a secret to an attacker's server
Controls & guardrails that address this
444 proposedGuardrails across the risks mapped to LLM08:2025, grouped by control function. Filter by control category below.
Design strict RBAC on training data repositories at design stage. Define approved contributor list and approval workflow.
Apply anomaly detection on the training data ingestion pipeline to identify poisoned or tampered batches.
Define and approve the source allow-list and write-time scanning during build. Prove non-allow-listed and injection-bearing writes are rejected before go-live.
source: OWASP Top 10 for LLM Apps LLM04:2025 Data and Model Poisoning, LLM08:2025 Vector and Embedding Weaknesses; NIST SP 800-53 AC-3 / SI-7Cleaning documents as they enter the library — stripping hidden text and active instructions — and only ingesting from trusted places.
Knowing exactly where the model came from, checking it hasn't been swapped, and testing its behaviour before going live.
Establish data transfer and storage policy for AI training data. Enforce approved storage locations from point of collection.
Implement DLP controls in the data acquisition environment to prevent unauthorised extraction or transfer of training data.
Enforce data handling policy in the build environment. Require explicit approval for any data transfers outside the environment.
Configure DLP controls in the build environment to block training data from leaving approved boundaries.
Conduct a privacy risk assessment at use case design stage. Determine if a DPIA is required before data acquisition.
Apply S1-defined privacy controls during data acquisition: verify consent, minimise data, anonymise personal data.
Apply anonymisation and masking controls to personal data before use in model training. Validate de-identification effectiveness.
Apply Privacy by Design in model architecture using differential privacy or federated learning where technically feasible.
Publish the privacy notice and confirm consent management is operational before go-live.
Define and sign off a purpose-to-data-source matrix with lawful basis at intake. Make it the approved baseline for runtime enforcement.
source: NIST AI RMF MAP 1.1 / MANAGE 2.2 (context and intended purpose); NIST SP 800-53 AC-4 / AC-3 (purpose-based access enforcement)Sign zero-retention/no-training terms with each model provider and obtain DPO sign-off on the data flow before enabling any endpoint.
source: OWASP Top 10 for LLM Apps LLM02:2025 Sensitive Information Disclosure; NIST SP 800-53 SC-8 / AC-4 (information flow enforcement)Implement output filters to detect and suppress quasi-identifying attribute combinations in model responses.
Propagate source ACLs and classification labels onto every chunk at ingestion. Reject documents whose entitlements cannot be resolved.
source: OWASP Top 10 for LLM Apps LLM02:2025 Sensitive Information Disclosure; NIST SP 800-53 AC-3 / AC-4 Information Flow Enforcement; OWASP Agentic AI Threats & Mitigations (privilege compromise)Scan every model response inline with DLP before delivery; redact or block PII, PAN and MNPI matches. Keep the rule set version-controlled.
source: OWASP Top 10 for LLM Apps LLM02:2025 Sensitive Information Disclosure; NIST SP 800-53 SC-7(10) Prevent Exfiltration, SI-4An egress allowlist only contains exfiltration if no allowlisted destination can be coerced into fetching an attacker-controlled URL. Audit each allowlisted domain/endpoint for image-search / link-preview / URL-fetch features (SSRF proxies), and either remove them, pin them to fixed paths, or route them through an inspecting forward proxy. Pair with finishing output sanitization before render so no auto-fetch fires un-inspected.
source: Case study: searchleak-copilot (Varonis Threat Labs, CVE-2026-42824; reported by Microsoft as critical, mitigated server-side ~Jun 2026)Controlling where the AI can send data, so secrets can't be quietly shipped to a stranger's address or website.
Making sure the library only returns documents this particular user is allowed to see.
Giving the agent only the keys it needs for the current task, not a master key to everything.
Making sure the machinery running the model — and the template used to stamp out new agents — is the real, unmodified version, and that one user's data can't leak into another's through shared shortcuts.
Conduct a data poisoning threat assessment at design stage. Identify likely attack vectors and assign risk ratings.
Simulate data poisoning attacks (backdoor, label flipping, gradient-based) to assess model resilience before deployment.
Verify a signed attestation and content hash on every dataset shard at ingestion. Reject unsigned or hash-mismatched data before it reaches the training pipeline.
source: MITRE ATLAS AML.M0007 (Sanitize Training Data), AML.M0014 (Verify ML Artifacts); NIST SP 800-53 SI-7 Software, Firmware, and Information Integrity, SR-4 ProvenanceGate every model promotion on backdoor-trigger probes and a behavioral diff against the approved baseline. Block release on significant regressions or trigger-pattern anomalies.
source: MITRE ATLAS AML.M0014 (Verify ML Artifacts), AML.M0019 (Red Teaming); NIST AI RMF MANAGE 2.2 and MEASURE 2.7For assistants that retrieve from the open web, rank and weight results by authenticated source reputation and independence — not just relevance / query-form match — so an anonymous, newly-registered, single-purpose site cannot become authoritative grounding. Run coordinated-inauthentic-content detection (look-alike site clusters, missing byline / legal entity, passages engineered for query-agnostic retrieval) and quarantine suspect sources. Surface per-citation provenance so users can see and discount low-trust sources. Does not defeat a well-resourced GEO campaign outright; it raises the cost and shrinks the yield.
source: Case study: hanover-institute-generative-engine-poisoning (per Politico investigation as reported by Arab News and Calcalist; FARA-disclosed funding/orchestration attributed to the reporting, not independently confirmed)Keeping a label on every document saying where it came from, so you can tell trusted company docs from random web text.
Live dashboards and alarms that notice unusual behaviour — spikes in errors, weird actions, sudden data access.
Monitor production for anomalous data transfers in real time. Alert on any transfer outside approved data flow boundaries.
Tag personal data with subject identifiers at ingestion and maintain an artefact inventory map of every store it reaches. Keep lineage current so erasure can propagate.
source: NIST AI RMF MANAGE 4.1 (post-deployment response); NIST SP 800-53 SI-12 Information Management and Retention, PT-2/PT-3 (personal data processing)Seed registered canary records into the fine-tuning corpus during data preparation. Control the seed manifest so canaries stay traceable and tamper-proof.
source: MITRE ATLAS AML.T0024 (Exfiltration via ML Inference API), AML.T0024.000 (Infer Training Data Membership); NIST AI RMF MEASURE 2.7A screen that reads incoming messages and blocks obvious attacks or banned topics before the model sees them.
Recording everything — questions, documents fetched, actions taken — so you can investigate when something goes wrong.
Penetration test the training data pipeline to identify injection points and access control weaknesses.
Scan every ingestion batch with spectral-signature and clustering detectors before training. Quarantine flagged clusters for human review against documented thresholds.
source: MITRE ATLAS AML.M0007 (Sanitize Training Data); OWASP Top 10 for LLM Apps LLM04:2025 Data and Model Poisoning; NIST AI RMF MEASURE 2.7Tie the persistent-memory lifecycle to identity state so that standard remediation actually ends a compromise. On password reset, credential rotation, session revocation or device re-enrollment, invalidate (or quarantine for re-review) memory entries — especially entries whose provenance traces to summarised untrusted content — so a planted standing instruction cannot outlive the reset. Pair with write-path validation/provenance so instruction-shaped memory-writes from web content are caught on the way in.
source: Case study: cosnitch-copilot-personal-oneclick (Varonis Threat Labs, CVE-2026-24301; reportedly patched 18 Aug 2026, no evidence of abuse)Continuously correlate live agent-memory writes against output behaviour to flag drift, then quarantine and roll back the suspected-poisoned memory record across all affected sessions.
source: Interactive-control reconciliation: ctrl-memory-quarantine (partial coverage)Monitor for privacy incidents in production including personal data appearing in outputs. Notify regulators within required timeframes.
Tag every memory and vector record with subject-id and retention class; partition stores per tenant/user. Prove the erasure and isolation paths in testing before release.
source: OWASP Agentic AI Threats & Mitigations (memory/knowledge-base privacy); NIST SP 800-53 SI-12 Information Management and RetentionTest de-identification approach against known re-identification attacks (quasi-identifier linkage, singling-out). Remediate if risk is high.
Conduct periodic data leakage audits including training data memorisation testing. Escalate confirmed leakage incidents to PDPA notification process.
Implement tamper-evident capture of prompts, outputs, and version state during build. Verify a full incident timeline can be reconstructed before go-live.
source: NIST SP 800-86 Guide to Integrating Forensic Techniques into Incident Response; ISO/IEC 27037 evidence handling; NIST SP 800-61r2 (Detection & Analysis – evidence handling)Run agent tool calls in a network-restricted sandbox behind a deny-by-default egress allow-list. Require security approval for any destination added.
source: OWASP Top 10 for LLM Apps LLM02:2025 Sensitive Information Disclosure; OWASP Agentic AI Threats & Mitigations (tool-misuse / exfiltration); NIST SP 800-53 SC-7 Boundary Protection / AC-4