🔍AI RiskAtlas
← Real-world cases
Case study

SesameOp: backdoor abuses the OpenAI Assistants API as covert command-and-control

Real-world incident03 Nov 2025🗺️ Model / Package Supply Chain

Microsoft's incident-response team found a .NET backdoor that hid its command-and-control channel inside a legitimate OpenAI Assistants API account, fetching encrypted commands stored as Assistant messages — turning an LLM provider's API into stealth attacker infrastructure.

Root cause — why it happened

Attackers had already broken into a company's network and wanted a quiet way to keep sending commands to the malware they left behind. Instead of phoning home to an obvious attacker server (which security tools watch for), they used a normal, trusted AI service — OpenAI's Assistants API — as a hidden mailbox. They left encrypted commands sitting in that account; the malware checked the mailbox, ran the commands, and dropped the answers back in the same place. To the company's firewall it just looked like ordinary traffic to a popular AI provider, so nothing seemed wrong.

Risks this case illustrates

Named in the standard (OWASP/ATLAS/NIST) lens. Click a highlighted component in the diagram below to see which risks attach where.

How it unfolded

Untrusted supply chainYour infrastructureuploads artefactpull by name / taginstall packageload (code can run!)servesstores encrypted command as AssistantXOR-decodes & runspolls for 'Payload'posts results ('Result')🌐Publisher(maybe🏪Model / PackageRegistry🧬Downloadedmodel / package🏗️Your build /serving stack🧠Your deployedmodel🌐Attackeroperator🔌OpenAIAssistants API🔧.NET backdooron compromised🎛️Obfuscatedloader
InstructionsDataActionsControl / decisionFeedback / logs
👆 Click a component to inspect its risks
SetupStep 1 / 7

A foothold already exists on the network

Before any of the AI trickery, the attackers were already inside the victim's environment — reportedly for several months. The clever part of this story isn't how they got in; it's how they stayed in touch with their malware without getting noticed.

Step 1 / 7

Controls & guardrails — what would have stopped it

The fix that actually bites: stop treating popular AI services as automatically safe. Only let the machines that genuinely need them connect, watch how that traffic behaves, and you'll spot a quiet machine that suddenly starts chatting with an AI API at odd hours. Blocking one service isn't enough — attackers just move to the next trusted one.

Preventive
  • Egress allowlisting & DLP on tool arguments

    Allowlists fight an open-ended channel; legitimate-but-broad destinations (any URL fetch, any email) are hard to constrain without breaking usefulness. Encoding can evade naive DLP.

  • Least-privilege identity & scoped credentials
    addressesTool Misuse

    Doesn't prevent manipulation — only caps its reach. Hard to get right operationally; over-broad scopes are the common real-world failure.

  • Governance: risk assessment, red-teaming & incident response

    Process reduces likelihood and speeds recovery but executes no technical control itself; weak follow-through makes it theatre.

Detective
  • Runtime monitoring & anomaly detection
    addressesTool Misuse

    Detects the anomalous, not the novel-but-subtle; high false-positive rates cause alert fatigue. Always a step behind a sufficiently quiet attacker.

  • Full-trace audit logging
    addressesTool Misuse

    Logging is forensic, not preventive — it explains harm after the fact. Useless if no one reviews it or if the materialised context isn't captured.

Corrective
  • Loop/cost circuit-breakers & consistency checks

    Thresholds are blunt — too tight breaks legitimate long tasks, too loose lets damage accrue first. Catches runaway dynamics, not a single well-formed bad decision.

  • Governance: risk assessment, red-teaming & incident response

    Process reduces likelihood and speeds recovery but executes no technical control itself; weak follow-through makes it theatre.

Lessons

  • A legitimate, high-reputation AI API can be repurposed as covert C2 — destination reputation is not a trust signal once the destination is programmable.
  • This was misuse of built-in functionality, not a platform vulnerability; the defensive gap was on the victim's egress side, not in the provider.
  • When commands and exfiltrated data are encrypted before leaving, on-wire DLP sees nothing — detection must be behavioural (which host, what cadence, what entropy).
  • Treat outbound traffic to AI/SaaS APIs as a monitored egress channel: allowlist initiating hosts, baseline usage, and log it for forensics.
  • Provider-side account/key revocation is a backstop you don't control; deprecating one API closes one relay but the living-off-trusted-services technique generalises.

Proposals & gaps this case surfaced

Non-destructive suggestions for the library — proposed, not adopted.

✚ proposed guardrailEgress monitoring & allowlisting of outbound AI/LLM-provider API traffic from enterprise endpoints (living-off-trusted-services C2)Agent Runtime Safety & Containment

Treat outbound connections to AI/LLM provider APIs as a monitored egress channel: allowlist which hosts may reach them, baseline usage (cadence, entropy, initiating process), and alert on out-of-profile traffic — because a high-reputation destination cannot itself be trusted once it is programmable and can relay encrypted commands/results.

coverage gapTool Misuse

This case shows a gap: we usually worry about attackers tricking an AI, or shipping a poisoned model. Here a normal AI service was turned into the attacker's secret mailbox. 'A trusted AI/SaaS API used as covert attacker infrastructure' isn't really covered as its own risk — and neither is the simple defence of watching outbound traffic to those services.

These surface as proposals across the Control Library and Risk Taxonomy; adopt them by hand when ready.

AI RiskAtlas is an educational model of how GenAI & agentic systems work and fail. Architectures and payloads are illustrative and simplified for learning — not operational guidance. Real-world cases are summarised from public reporting.

Sources & further reading →·Built by Shi Yuan ↗