๐Ÿ”AI RiskAtlas
โ† Risk Taxonomy
#44

Disruption to connected systems

IMDA agentic
Risk taxonomy

Definition

Actions taken by an AI agent disrupt or damage connected systems it interacts with (production codebases, third-party APIs, downstream agents) through compromise, malfunction or excessive load. In multi-agent settings, faults can cascade across coordinated agents and amplify the impact.

Interactive deep-dive

This risk surfaces under more than one interactive treatment โ€” each with its own technical detail, attack surface, detection signals, and scenarios.

โ˜… Suggested sub-risks โ€” not yet in your taxonomy

Granular vectors recommended under this risk.

Exposed / unauthenticated inference endpoint (LLMjacking)โ–ถ interactive scenario โ†’

A self-hosted inference/serving or MCP endpoint (e.g. Ollama, an OpenAI-compatible API, or an access-control-less MCP server) is reachable from an untrusted network without authentication, allowing third parties to hijack the inference compute (resale/denial-of-wallet/mining), read prompt and conversation state, and โ€” via co-located over-privileged tools โ€” pivot into connected systems.

Controls & guardrails that address this

142 proposed

Grouped by control function, with the AI lifecycle stage(s) to apply each and the other risks it addresses. Filter by control category below.

Control category
Preventive ยท 3
Dependency integration safety contracts with schema validation and version pinning

Register a safety contract per integration โ€” pinned version, schemas, side-effect class, latency/error envelope. Gate onboarding on contract review and sign-off.

source: OWASP Top 10 for LLM Apps LLM05:2025 Improper Output Handling; NIST SP 800-53 SA-9 External System Services
Lifecycle stages3 โ€“ Onboarding, Build & Review5 โ€“ Usage, Monitoring & Change
Change-freeze and blackout-window enforcement on agent-initiated changes

Wire the agent tool layer to the CAB calendar at deployment. Test that a declared freeze blocks mutating calls before go-live.

source: NIST SP 800-53 CM-3 Configuration Change Control, CM-5 Access Restrictions for Change; ITIL change-freeze practice
Lifecycle stages4 โ€“ Deployment5 โ€“ Usage, Monitoring & Change
Admission control on the inference & MCP serving plane: authenticate and network-segment every self-hosted inference/serving and MCP endpointโœš proposed

Require authN/authZ on every inference API and MCP server, bind to private interfaces / front with a gateway, enforce network policy (no public exposure by default), and scope MCP tools to least privilege โ€” so an exposed endpoint cannot be hijacked for compute resale, prompt/history exfiltration, or lateral movement. Pair with continuous asset discovery so endpoints can't drift back to an open default.

source: Case study: operation-bizarre-bazaar-llmjacking (Pillar Security, 28 Jan 2026)
Lifecycle stage4 โ€“ Deployment & Serving
Detective ยท 1
Cross-agent consensus and consistency monitoring to detect sycophantic agreement and error amplificationโœš proposed

Run consistency and consensus checks across agent or model outputs to flag low-diversity agreement and amplifying error patterns, escalating or breaking the run before sycophantic convergence cascades into action.

source: Interactive-control reconciliation: ctrl-circuit-breaker (partial coverage)
Lifecycle stage5 โ€“ Usage, Monitoring & Change
Corrective ยท 10
Non-production-by-default execution environment with explicit production promotion gate

Bind the agent's default execution target to non-production environments at design time. Require a separately approved promotion configuration for any production-connected target.

source: NIST SP 800-53 SC-7 Boundary Protection, CM-2 Baseline Configuration; OWASP Agentic AI Threats & Mitigations (cascading failures)
Lifecycle stages1 โ€“ Use Case Context & Design4 โ€“ Deployment
Graceful degradation and manual-fallback workflow on dependency unavailability

Map every dependency failure mode to a defined safe behaviour at design. Require architecture sign-off on the fallback specification before build.

source: NIST SP 800-53 CP-12 Safe Mode, SC-5 Denial-of-Service Protection; NIST AI RMF MANAGE 4.1 (post-deployment response/recovery)
Lifecycle stages1 โ€“ Use Case Context & Design4 โ€“ Deployment
Blast-radius scoping and environment isolation per agent task

Run each agent task in an isolated, network-segmented sandbox scoped to the task's exact needs. Gate onboarding on fault-injection tests proving containment.

source: NIST SP 800-53 SC-7 Boundary Protection, SC-39 Process Isolation; OWASP Agentic AI Threats & Mitigations (sandboxing/containment)
Lifecycle stages3 โ€“ Onboarding, Build & Review5 โ€“ Usage, Monitoring & Change
Cross-agent cascading-failure detection and orchestrator-level circuit breaking

Build tracing, detection rules and breaker thresholds into the orchestrator. Prove via fault-injection tests that a failing agent is quarantined within target before release.

source: OWASP Agentic AI Threats & Mitigations (cascading failures); Cloud Security Alliance MAESTRO (multi-agent threat modelling)
Lifecycle stages3 โ€“ Onboarding, Build & Review5 โ€“ Usage, Monitoring & Change
Idempotent action design with transactional rollback and pre-action snapshots

Engineer mutating actions with idempotency keys, transactions and pre-change snapshots; stage writes rather than committing directly. Gate release on tested dedup and rollback within RPO.

source: NIST SP 800-53 CP-9 System Backup, CP-10 System Recovery and Reconstitution; established idempotency / safe-write engineering practice
Lifecycle stages3 โ€“ Onboarding, Build & Review5 โ€“ Usage, Monitoring & Change
Rate, quota, and budget circuit breakers on outbound calls to connected systems

Cap each agent's rate, volume, concurrency, and spend per downstream dependency. Trip the breaker and fail closed when a ceiling is crossed.

source: NIST SP 800-53 SC-5 Denial-of-Service Protection, SC-6 Resource Availability; OWASP Top 10 for LLM Apps LLM10:2025 Unbounded Consumption
Lifecycle stages4 โ€“ Deployment5 โ€“ Usage, Monitoring & Change
Loop, recursion-depth, and iteration caps with runaway-loop detection

Enforce hard caps on iterations, depth, wall-clock, and cost per agent run. Terminate the run on cap breach or detected loop signatures.

source: OWASP Top 10 for LLM Apps LLM10:2025 Unbounded Consumption; OWASP Agentic AI Threats & Mitigations (cascading failures)
Lifecycle stages4 โ€“ Deployment5 โ€“ Usage, Monitoring & Change
Staged rollout with canary release and automated rollback on health-signal breach

Roll out agent changes via shadow and canary stages gated on connected-system health signals. Auto-halt and roll back to last known-good on threshold breach.

source: NIST SP 800-53 SI-2 Flaw Remediation, CM-3 Configuration Change Control; established progressive-delivery / canary practice
Lifecycle stages4 โ€“ Deployment5 โ€“ Usage, Monitoring & Change
Tiered kill-switch with per-agent, per-tool, and per-dependency containment scope

Deploy revocation, tool-cutoff and fleet-halt mechanisms with the release. Test every tier end-to-end and record time-to-effect before go-live.

source: OWASP Agentic AI Threats & Mitigations (kill-switch / containment); NIST AI RMF MANAGE 2.4 (mechanisms to supersede, disengage, or deactivate AI systems)
Lifecycle stages4 โ€“ Deployment5 โ€“ Usage, Monitoring & Change
Rollback and restore-to-known-good recovery procedure for AI services

Register each release as a restorable known-good baseline and rehearse rollback at the release gate. Block promotion without a tested restore.

source: ISO/IEC 27031 ICT readiness for business continuity; NIST SP 800-34r1 Contingency Planning (Recovery phase); NIST AI RMF MANAGE 2.4 (mechanisms to supersede/disengage/deactivate)
Lifecycle stages4 โ€“ Deployment5 โ€“ Usage, Monitoring & Change
Open these in the Control Library โ†’

Other risks in Robustness & Stability

AI RiskAtlas is an educational model of how GenAI & agentic systems work and fail. Architectures and payloads are illustrative and simplified for learning โ€” not operational guidance. Real-world cases are summarised from public reporting.

Sources & further reading โ†’ยทBuilt by Shi Yuan โ†—