Definition
Data is biased against, or unevenly represents, certain individuals or groups of individuals, which can produce biased model outputs.
Controls & guardrails that address this
16Grouped by control function, with the AI lifecycle stage(s) to apply each and the other risks it addresses. Filter by control category below.
Conduct fairness impact assessment at use case intake. Require governance sign-off on demographic coverage requirements before data acquisition.
Select modelling algorithm based on bias risk profile. Prefer algorithms with lower sensitivity to demographic distribution shifts.
Design separate model modules for distinct demographic populations where data characteristics diverge materially.
Apply adversarial debiasing or fairness constraints during model training. Validate against fairness metrics before sign-off.
Tune hyperparameters with fairness-aware search objectives. Reject configurations with demographic disparity exceeding threshold.
Fine-tune on a curated, representative dataset verified for demographic balance. Document coverage breakdown before training.
Calibrate decision thresholds per demographic group to equalise error rates. Validate calibration before deployment sign-off.
Apply post-processing adjustments (re-ranking, score recalibration) to correct fairness gaps identified in validation.
Design system prompts to include explicit fairness requirements: instruct the model to avoid stereotyping and demographic assumptions.
Disclose to all users at deployment that model outputs may reflect training data biases. Include specific limitation caveat.
Conduct comprehensive fairness validation across demographic groups before deployment. Treat material disparity as a blocking defect.
Screen training data for demographic gaps using automated pipeline checks. Reject batches failing representation thresholds.
Execute adversarial bias testing using targeted demographic test cases before deployment.
Conduct structured human expert review of model outputs stratified across demographic groups before deployment.
Continuously monitor fairness metrics across demographic groups in production. Trigger model review when bias drift is detected.
Monitor fairness metric trends by demographic group in production. Use feedback to drive targeted debiasing in model updates.