Retrieval / RAG

Design strict RBAC on training data repositories at design stage. Define approved contributor list and approval workflow.

Apply anomaly detection on the training data ingestion pipeline to identify poisoned or tampered batches.

Define and approve the source allow-list and write-time scanning during build. Prove non-allow-listed and injection-bearing writes are rejected before go-live.

Establish data transfer and storage policy for AI training data. Enforce approved storage locations from point of collection.

Implement DLP controls in the data acquisition environment to prevent unauthorised extraction or transfer of training data.

Enforce data handling policy in the build environment. Require explicit approval for any data transfers outside the environment.

Configure DLP controls in the build environment to block training data from leaving approved boundaries.

Conduct a privacy risk assessment at use case design stage. Determine if a DPIA is required before data acquisition.

Apply S1-defined privacy controls during data acquisition: verify consent, minimise data, anonymise personal data.

Apply anonymisation and masking controls to personal data before use in model training. Validate de-identification effectiveness.

Apply Privacy by Design in model architecture using differential privacy or federated learning where technically feasible.

Publish the privacy notice and confirm consent management is operational before go-live.

Define and sign off a purpose-to-data-source matrix with lawful basis at intake. Make it the approved baseline for runtime enforcement.

Sign zero-retention/no-training terms with each model provider and obtain DPO sign-off on the data flow before enabling any endpoint.

Implement output filters to detect and suppress quasi-identifying attribute combinations in model responses.

Propagate source ACLs and classification labels onto every chunk at ingestion. Reject documents whose entitlements cannot be resolved.

Scan every model response inline with DLP before delivery; redact or block PII, PAN and MNPI matches. Keep the rule set version-controlled.

Implement confidence scoring to communicate output certainty alongside each result. Calibrate before deployment.

Define model accuracy acceptance criteria aligned to business requirements before validation commences.

Implement counterfactual explanation to show users what changes would alter the model's output.

Communicate model accuracy, known limitations, and uncertainty to users in the production interface at launch.

Monitor production accuracy continuously against the validated baseline. Trigger model review when accuracy degrades.

Specify a RAG architecture at design stage for factual domains. Define grounding requirements and acceptable hallucination thresholds.

Evaluate foundation model candidates on hallucination benchmarks at design stage. Select models with lowest documented rates.

Design system prompts to instruct the model to acknowledge uncertainty, cite sources, and refuse when knowledge is insufficient.

Fine-tune on a curated, domain-specific dataset to improve factual accuracy. Validate hallucination rates pre/post fine-tuning.

Configure conversation controls at deployment to restrict the model to approved topic domains and escalate off-topic queries.

Establish acceptable hallucination rate thresholds and grounding requirements as policy before build. Assign a named risk owner.

Configure tiered HITL review for high-stakes factual outputs with defined trigger criteria and reviewer SLAs.

Calibrate the initial entropy threshold on a knowledge-boundary dataset; approve sampling design and thresholds per risk tier.

Map each fact class to a designated tool, embed the no-ungrounded-assertion prompt, and gate build review on grounding tests passing.

Resolve every emitted citation against the approved corpus and verify span-level entailment before display. Strip or withhold claims with fabricated or non-entailing references.

Define third-party AI accountability requirements before vendor engagement. Embed in RFP and contract specifications.

Conduct AI governance due diligence on third-party providers at selection stage. Reject providers failing minimum maturity.

Require third-party providers to submit model cards, validation reports, and security documentation before integration.

Enforce ongoing third-party accountability obligations including incident notification and periodic performance reporting.

Conduct independent performance and compliance monitoring of third-party AI components. Escalate when SLA or compliance obligations are missed.

Allocate every control in a shared-responsibility matrix and flow down regulatory obligations in contract at onboarding. Gate approval on initial assurance artefacts.

Declare all planned training and test data sources at use case intake, with provenance status for each.

Plan the interpretability approach at design stage to ensure source provenance can be traced and disclosed to users.

Document actual provenance for each data source during collection: origins, methods, timestamps, custodian identity.

Conduct a data poisoning threat assessment at design stage. Identify likely attack vectors and assign risk ratings.

Simulate data poisoning attacks (backdoor, label flipping, gradient-based) to assess model resilience before deployment.

Verify a signed attestation and content hash on every dataset shard at ingestion. Reject unsigned or hash-mismatched data before it reaches the training pipeline.

Gate every model promotion on backdoor-trigger probes and a behavioral diff against the approved baseline. Block release on significant regressions or trigger-pattern anomalies.

Monitor production for anomalous data transfers in real time. Alert on any transfer outside approved data flow boundaries.

Tag personal data with subject identifiers at ingestion and maintain an artefact inventory map of every store it reaches. Keep lineage current so erasure can propagate.

Seed registered canary records into the fine-tuning corpus during data preparation. Control the seed manifest so canaries stay traceable and tamper-proof.

Define and execute a domain-specific hallucination test suite before deployment. Treat hallucination rate above threshold as a blocking defect.

Construct synthetic evaluation datasets for knowledge-boundary scenarios. Use to validate model refusal behaviour.

Calibrate the groundedness threshold against the hallucination test suite pre-release; sign off the threshold in the validation pack.

Build and baseline the golden-set suite against the vendor model before go-live. Sign off thresholds with the model risk owner as a release condition.

Re-verify hashes and signatures on every vendor model update before promotion. Reconcile deployed artifacts against the AIBOM on a set cadence.

Penetration test the training data pipeline to identify injection points and access control weaknesses.

Scan every ingestion batch with spectral-signature and clustering detectors before training. Quarantine flagged clusters for human review against documented thresholds.

Monitor for privacy incidents in production including personal data appearing in outputs. Notify regulators within required timeframes.

Tag every memory and vector record with subject-id and retention class; partition stores per tenant/user. Prove the erasure and isolation paths in testing before release.

Test de-identification approach against known re-identification attacks (quasi-identifier linkage, singling-out). Remediate if risk is high.

Conduct periodic data leakage audits including training data memorisation testing. Escalate confirmed leakage incidents to PDPA notification process.

Implement tamper-evident capture of prompts, outputs, and version state during build. Verify a full incident timeline can be reconstructed before go-live.

Run agent tool calls in a network-restricted sandbox behind a deny-by-default egress allow-list. Require security approval for any destination added.

Use production feedback (user corrections, fact-check failures) to drive periodic RLHF cycles. Update model when error rates trend upward.

Require user-facing interfaces to disclose Gen AI limitations and hallucination risk before go-live.

Score every RAG answer for groundedness before release; block, fall back, or escalate responses below the faithfulness threshold.

Sample multiple generations for high-stakes queries and abstain, fall back, or escalate when semantic entropy exceeds the calibrated threshold.

Design all vendor model access behind a gateway with pinned versions, a second-vendor fallback, and a documented exit plan. Gate architecture sign-off on no single-sourcing.

Verify every third-party model artifact against its AIBOM hashes and signatures before load. Fail the build on any unverified artifact.

Review independent vendor assurance on cadence, log gaps, and track remediation. Keep the shared-responsibility matrix current so every control has an owner.