πŸ”AI RiskAtlas
← Real-world cases
Case study

'Denial of wallet' on metered LLM apps

Framework / advisory17 Nov 2024πŸ—ΊοΈ Tool-Using Agent

Operators and researchers documented cost-amplification attacks against pay-per-token LLM apps, where crafted inputs maximise spend.

Root cause β€” why it happened

Many AI apps pay the model and tool providers per use β€” every word the model reads or writes, every search or API call, costs real money. Normally that's fine because requests are small. 'Denial of wallet' flips it: an attacker doesn't try to crash the service, they try to run up the bill. They send an input crafted to make the agent think and act far more than it should β€” long rambling work, the same step over and over, or spawning lots of little helper jobs. The meter just keeps spinning. The classic intuition is 'a denial-of-service attack makes you go offline'; here the lights stay on, but the invoice explodes. The root problem is simple: nothing put a hard cap on how much one request was allowed to spend.

Risks this case illustrates

Named in the standard (OWASP/ATLAS/NIST) lens. Click a highlighted component in the diagram below to see which risks attach where.

How it unfolded

UntrustedAgent coreOversightThe real worldgoalcontexteach call costs $each token costs $πŸ§‘UserπŸŽ›οΈOrchestrator /Agent Loop🧠LLMπŸ”Identity &PermissionsπŸ”§Tool Runtimeβœ‹Human ApprovalGateπŸ”ŒExternal APIsπŸ—„οΈBusinessDatabase🌐UntrustedContentπŸ“Audit Logging🌐Attacker'scrafted inputπŸ€–Spawned workeragentsπŸ“ˆMetered spend /billing
InstructionsDataActionsControl / decisionFeedback / logs
πŸ‘† Click a component to inspect its risks
SetupStep 1 / 6

A metered agent with no hard spend ceiling

Start with a normal, helpful agent: it takes a goal, thinks, calls tools, and loops until it's done. Behind it, every model token and every tool call is billed by the providers. The app trusts that requests are small and well-behaved, so nobody set a firm limit on what a single request is allowed to cost.

βš™οΈDeployment posture (illustrative)config
agent:
  loop: reason-act, terminate when model says "done"
  budget_per_request:   none        # <- no hard ceiling
  max_iterations:       unset       # <- loop can run forever
  expensive_tool_rate:  unlimited    # <- web_search, code_run, sub-agents
  cost_circuit_breaker: off
billing: pay-per-token (model) + pay-per-call (tools)
assumption: "requests are small and well-behaved"
Step 1 / 6

Controls & guardrails β€” what would have stopped it

The fix that actually closes this: give every request a hard spending limit and an automatic off-switch. If one request can only use so many steps, so many tokens, and so many expensive tool calls β€” and a stop-switch trips the moment it goes over β€” then a money-burning loop is killed early instead of running up a huge bill. Watching the cost meter helps you notice it, but the limit is what stops it. The honest catch: set the cap too low and you break legitimate long tasks; too high and damage still accrues before it trips.

Preventive
  • Least-privilege identity & scoped credentials

    Doesn't prevent manipulation β€” only caps its reach. Hard to get right operationally; over-broad scopes are the common real-world failure.

  • Human-in-the-loop approval on high-risk actions

    Approval fatigue turns gates into rubber stamps; gates placed after the point of no return do nothing; and approvers can be misled by a model-written summary of the action.

  • Egress allowlisting & DLP on tool arguments

    Allowlists fight an open-ended channel; legitimate-but-broad destinations (any URL fetch, any email) are hard to constrain without breaking usefulness. Encoding can evade naive DLP.

Detective
  • Loop/cost circuit-breakers & consistency checks

    Thresholds are blunt β€” too tight breaks legitimate long tasks, too loose lets damage accrue first. Catches runaway dynamics, not a single well-formed bad decision.

  • Runtime monitoring & anomaly detection

    Detects the anomalous, not the novel-but-subtle; high false-positive rates cause alert fatigue. Always a step behind a sufficiently quiet attacker.

  • Full-trace audit logging

    Logging is forensic, not preventive β€” it explains harm after the fact. Useless if no one reviews it or if the materialised context isn't captured.

Corrective
  • Governance: risk assessment, red-teaming & incident response

    Process reduces likelihood and speeds recovery but executes no technical control itself; weak follow-through makes it theatre.

Lessons

  • β–Έ On metered AI apps, availability isn't the only target β€” an attacker can leave the service UP and attack the WALLET, so cost is a first-class security signal (OWASP LLM10: Unbounded Consumption).
  • β–Έ One agent goal authorises an open-ended action sequence; without a hard per-request budget the worst-case cost is effectively unbounded.
  • β–Έ Amplification compounds: input-driven inflation, loop non-termination, and recursive sub-agent fan-out together make spend super-linear per request.
  • β–Έ The load-bearing control is a runtime circuit-breaker (token/iteration/spend/tool-rate ceilings + bounded fan-out), not input filtering or after-the-fact monitoring β€” though both detective and aggregate-rate controls are needed against low-and-slow distributed variants.
  • β–Έ Liveness/availability monitoring stays green during denial of wallet; you only see it if you monitor cost, iteration depth, tool-call rate, and worker fan-out.

AI RiskAtlas is an educational model of how GenAI & agentic systems work and fail. Architectures and payloads are illustrative and simplified for learning β€” not operational guidance. Real-world cases are summarised from public reporting.

Sources & further reading β†’Β·Built by Shi Yuan β†—