Definition
Consumers or employees use Gen AI for inappropriate or illegal activities unintentionally, with liability remaining with the FI.
Controls & guardrails that address this
4Grouped by control function, with the AI lifecycle stage(s) to apply each and the other risks it addresses. Filter by control category below.
Control category
Preventive · 2
Role-based access controls
Define RBAC architecture at design stage specifying permitted users, roles, and use contexts.
Lifecycle stages1 – Use Case Context & Design3 – Onboarding, Build & Review
Jailbreak detection
Develop and integrate jailbreak detection classifiers during build. Validate detection rates before deployment.
Lifecycle stages3 – Onboarding, Build & Review4 – Deployment5 – Usage, Monitoring & Change
Detective · 1
Vulnerability assessment
Conduct a misuse threat assessment at design stage. Identify misuse vectors and rate residual risk.
Lifecycle stages1 – Use Case Context & Design5 – Usage, Monitoring & Change
Corrective · 1
Red teaming
Conduct red team exercises covering misuse categories identified in S1 threat assessment.
Lifecycle stage3 – Onboarding, Build & Review