Allocative Harm in Multi-User Arbitration
mediumAgency & toolsDefinition
When one AI agent serves many people at once, it has to decide whose request comes first or who gets a limited resource. If it does that unfairly — always favouring some users over others — it can quietly disadvantage whole groups, even without any single obvious error.
Where it attaches
The system components this risk arises at.
Detection signals
- ▸ Aggregate outcome or latency disparities across user cohorts or tenants
- ▸ A shared agent's prioritisation systematically favouring some users
- ▸ Resource starvation for low-priority or protected groups
- ▸ No per-cohort fairness monitoring on an agent serving many users
Controls & guardrails that address this
4Grouped by control function, with the AI lifecycle stage(s) to apply each and the other risks it addresses. Filter by control category below.
Pausing to ask a person before doing anything big or hard to undo — sending money, deleting data, emailing customers.
Live dashboards and alarms that notice unusual behaviour — spikes in errors, weird actions, sudden data access.
Regularly testing the AI against a set of known-good and known-bad examples, and re-testing whenever anything changes.
The organisational habits around the AI: assessing risks before launch, actively trying to break it, and having a plan for when something goes wrong.
Framework mappings
- MEASURE 2.11
- GOVERN 1.2