Poisoning the Well

An attacker edits the wiki; the assistant cites the lie back to everyone

Technique first revealed 29 Oct 2023

🗺️ RAG Knowledge Assistant Knowledge / Training Data Poisoning

RAG Knowledge Assistant

InstructionsDataActionsControl / decisionFeedback / logs

👆 Click a component to inspect

SetupStep 1 / 7

An open wiki, a helpful bot

The company runs an assistant that answers staff questions by reading the internal wiki. Employees love it because the answers come straight from official-looking company pages. The catch: almost anyone can edit that wiki, and one section is even open to outside contributors.

⚙️Wiki access policy (excerpt)config

space: it-knowledge-base
  edit: any-authenticated-employee
  review_before_publish: false
space: community-howtos   # public contributors welcome
  edit: anyone-with-link
  review_before_publish: false

indexing:
  crawler: every 30 min
  sources: [it-knowledge-base, community-howtos]
  provenance_tags: none        # <-- all chunks indexed as 'trusted'

← / → keys