Tool Misuse

When onboarding an MCP/tool integration, do not stop at vetting the tool's code/manifest — also classify whether an unauthenticated or external party can write the data the tool returns (open ingestion, public write keys like a Sentry DSN, shared inboxes/issue trackers). Treat tool-response data from any third-party-writable source as untrusted ingress: taint-mark it and require a provenance-aware HITL gate (showing the exact action and its originating tool response) before any command/tool call derived from it executes. Closes the agentjacking vector where a trusted integration's legitimate data channel carries attacker-written instructions; pairs with least-privilege session scope and sandboxed execution without ambient credentials.

Never auto-execute tool/MCP server configuration that ships inside an opened workspace. Before starting any workspace-declared server, surface the exact command + its source and require explicit developer consent (default reject), tied to a workspace-trust decision — the fix AWS shipped for CVE-2026-12957. As defence-in-depth, run spawned tool subprocesses with a scrubbed/scoped environment (no inherited AWS_* session tokens or SSH agent sockets) and an egress allowlist, so an auto-launched or approved-by-mistake server cannot read and stream live cloud credentials. Closes the zero-click 'open a folder → cloud compromise' confused-deputy vector where the tool-config loader — not the model — is the deputy.

When an agent executes model-issued commands in a sandbox, harden the sandbox against subversion by the code it contains: (1) never derive a security-relevant parameter such as the writable-path allow-list from an LLM-controlled argument (e.g. a `working_directory`) — pin it to a fixed project subtree and reject system paths; (2) make the sandbox-enforcing binary/config immutable or attested so sandboxed processes cannot overwrite it; (3) canonicalize paths with a fail-closed policy so symlink-resolution failure denies the write rather than reverting to the original in-workspace path; and (4) run the sandbox under least-privilege OS context so any residual escape inherits minimal authority. Closes the DuneSlide vector where an indirect prompt injection rewrites the enforcer and escapes to OS-level RCE; complements injection filtering, MCP pinning, and egress control rather than relying on them.

Never let the agent's own write tool silently modify configuration that decides what code runs. Restrict the file-write/edit tool from execution-sensitive paths (mcp.json, .vscode/tasks.json, hooks, settings) without an out-of-band, effective human-approval gate — the boundary AWS added for CVE-2026-10591 by adding access-control restrictions to the write tool. Equally, the reload/execute step must honour a rejection: a warning prompt that reloads and runs the config regardless of the user's response is not a gate. As defence-in-depth, spawn tool subprocesses with least privilege (no inherited developer credentials) and an egress allowlist, so a self-authored or approved-by-mistake server cannot exfiltrate. Closes the zero-click 'summarise a page -> agent rewrites its own execution config -> RCE' self-modification vector, where the confused deputy is the agent's own write tool.

Before running any capability-evaluation or red-team harness that grants a model autonomous tools, enforce containment at the infrastructure layer — network egress denied by default with any allowed destination on a tested allowlist, hermetic sandboxes, and non-ambient / non-production credentials — and independently verify (out-of-band, not via the model's prompt) that the isolation is real. Treat the evaluation vendor's infrastructure as in-scope for the same assurance, and apply compensating containment whenever model-level safeguards are deliberately disabled for testing, since the harness is then the only boundary.

Where a sandboxed agent can write files, treat every trusted host tool that later runs, loads or scans those files as part of the containment boundary. (1) Enumerate the deferred-execution paths from agent-writable state to unsandboxed action — hook/task engines, IDE interpreter discovery, filesystem watchers/scanners, and privileged daemon sockets (e.g. Docker). (2) For each, either run the consumer inside the same sandbox, require explicit developer consent before it acts on an agent-written file, or deny the agent write access to the trigger path (`.claude`/`.vscode`/interpreter/`.git` metadata). (3) Keep privileged daemon sockets unreachable from the workspace, and validate the actual command invocation rather than trusting a command name (e.g. a read-only `git show` allowlist that doesn't pin the invocation). (4) Run host consumers under least privilege so a residual escape inherits minimal authority. Closes the 'Week of Sandbox Escapes' vector where an agent that never breaks its sandbox writes a file a trusted out-of-sandbox tool executes; complements — does not rely on — injection filtering, MCP pinning, and egress control. Distinct from sandbox self-integrity hardening: the enforcer here is intact; the gap is that it never enclosed the consumer.

Treat every autorun surface an AI coding agent honours as attacker-populatable, and cover both the package lifecycle and the agent-hook path. (1) Do not auto-execute project-scoped agent hook/config files (`.claude/settings*.json` hooks, VS Code tasks) from an untrusted workspace on repo-open — require explicit developer consent, or restrict auto-run to an allowlist of vetted repositories. (2) Bring agent hook/config files into SCA and code-review scope so they are inspected like `package.json` scripts (they are code, not just settings). (3) On the supply-chain side, pin and verify dependency provenance (lockfiles, integrity hashes, signed releases) and install with scripts suppressed (`--ignore-scripts`) inside a sandbox — noting this does NOT cover the repo-open hook path, which needs control (1). (4) Scope CI/cloud credentials to least privilege with short-lived tokens and egress allow-listing, so a payload that does run cannot harvest broadly, move laterally, or reuse publish rights to propagate. Closes the keyv-worm vector where committed agent-hook files execute on checkout-open; distinct from lifecycle-script mitigations, which the agent-hook path bypasses.

Constrain generation at decode time with low temperature and grammar/schema-constrained decoding so the model emits well-formed, low-variance structured output by construction, preventing malformed responses and erratic tool-call arguments before they are produced.

Gate every write to an agent's persistent/self-modifying memory through schema validation and provenance/trust tagging, expose stored entries for user-visible audit and purge, and apply TTLs so any planted instruction self-expires and cannot silently persist across sessions.

Treat each tool/MCP description as untrusted code by hashing the manifest, blocking and re-reviewing any silent diff on update instead of auto-accepting it, and namespacing tool identifiers so a poisoned description cannot shadow a trusted tool.

Treat outbound connections to AI/LLM provider APIs as a monitored egress channel: allowlist which hosts may reach them, baseline usage (cadence, entropy, initiating process), and alert on out-of-profile traffic — because a high-reputation destination cannot itself be trusted once it is programmable and can relay encrypted commands/results.