Interface & Prompt Layer

Design the system prompt architecture with privilege separation and trust tier definitions at design stage.

Implement input sanitisation and injection detection filters covering known injection patterns and privilege escalation attempts.

Wrap all untrusted content in random delimiters and datamarking; instruct the model never to execute instructions inside the marked region. Gate release on injection eval results.

Benchmark the classifier on a labelled injection corpus and tune the decision threshold. Sign off the operating point before deployment.

Define content safety policy at use case design stage. Classify prohibited content types and set zero-tolerance thresholds.

Select a foundation model with documented RLHF or Constitutional AI safety training. Verify against toxicity benchmarks.

Implement multi-layer content moderation (input + output) validated against toxicity benchmarks. Escalate when filter bypass rates spike.

Maintain live HITL review for deployments serving vulnerable users or high-risk contexts. Escalate confirmed toxic outputs immediately.

Design system prompts to explicitly prohibit toxic, hateful, and harmful content generation.

Mandate AI risk awareness training for all use case sponsors and design team members before project kick-off.

Mandate AI risk training for all build and test personnel. Gate project participation on training completion.

Mandate human verification for high-stakes decisions where over-reliance risk is elevated. Review automation bias incidents quarterly.

Surface AI limitation warnings and over-reliance caveats in every production interaction. Update disclosures when model changes.

Require AI governance training for all personnel involved in data acquisition and processing before project participation.

Verify all deployment, operations, and customer-facing team members have completed AI risk training before launch.

Define AI identity disclosure policy at design stage. Specify when and how the system must identify itself as AI.

Plan consent and AI identity disclosure touchpoints in the user journey at design stage.

Design system prompts to explicitly prevent the model from claiming human-like identity or implying sentience.

Implement persistent AI identity disclosures in the UI (opening banner, inline notifications). Test before deployment.

Verify all AI identity disclosure elements are live, accurate, and prominently visible before go-live.

Monitor production for anthropomorphism incidents. Escalate complaints where users believed they were interacting with a human.

Apply post-training calibration (temperature scaling, isotonic regression) to align confidence scores with accuracy. Validate ECE before deployment.

Classify the use case by consequence-of-error severity at design stage. Define overconfidence risk tolerance accordingly.

Configure output filters at deployment to detect and rewrite responses with overconfidence markers (absolute certainty language).

Route high-confidence outputs in high-stakes use cases to human review. Flag for reviewer attention when certainty language is absolute.

Disclose to users at deployment that outputs may carry unwarranted confidence. Include specific caveat language in the UI.

Implement confidence scoring to communicate output certainty alongside each result. Calibrate before deployment.

Define model accuracy acceptance criteria aligned to business requirements before validation commences.

Implement counterfactual explanation to show users what changes would alter the model's output.

Communicate model accuracy, known limitations, and uncertainty to users in the production interface at launch.

Monitor production accuracy continuously against the validated baseline. Trigger model review when accuracy degrades.

Specify a RAG architecture at design stage for factual domains. Define grounding requirements and acceptable hallucination thresholds.

Evaluate foundation model candidates on hallucination benchmarks at design stage. Select models with lowest documented rates.

Design system prompts to instruct the model to acknowledge uncertainty, cite sources, and refuse when knowledge is insufficient.

Fine-tune on a curated, domain-specific dataset to improve factual accuracy. Validate hallucination rates pre/post fine-tuning.

Configure conversation controls at deployment to restrict the model to approved topic domains and escalate off-topic queries.

Establish acceptable hallucination rate thresholds and grounding requirements as policy before build. Assign a named risk owner.

Calibrate the initial entropy threshold on a knowledge-boundary dataset; approve sampling design and thresholds per risk tier.

Map each fact class to a designated tool, embed the no-ungrounded-assertion prompt, and gate build review on grounding tests passing.

Resolve every emitted citation against the approved corpus and verify span-level entailment before display. Strip or withhold claims with fabricated or non-entailing references.

Apply anomaly detection on the training data ingestion pipeline to identify poisoned or tampered batches.

Define and approve the source allow-list and write-time scanning during build. Prove non-allow-listed and injection-bearing writes are rejected before go-live.

Conduct ethical design review at intake specifically examining interface design for dark patterns.

Publish a prohibited dark pattern taxonomy and embed it as a design constraint before build.

Require HITL review for AI outputs in high-persuasion contexts (financial recommendations, healthcare advice).

Train PII-bearing models with DP-SGD under a documented epsilon/delta budget. Approve the budget against the enterprise epsilon-ceiling policy before training.

Strip raw logits, quantise confidence scores and block training-record echoes at the inference gateway. Keep the output-filter policy under change control.

Identify all groups at risk of adverse impact at use case intake. Register them in the affected group register.

Design separate model segments where adverse impact risk differs materially across population groups.

Set decision thresholds to meet acceptable adverse impact ratios across protected groups. Validate before deployment.

Apply post-processing adjustments (reject-option classification, score recalibration) to meet adverse impact targets.

Ensure HITL review pathways are live and tested for high-impact adverse decisions at go-live.

Maintain HITL review for all AI decisions with material adverse impact potential. Log all interventions and outcomes.

Conduct a prompt injection threat assessment at design stage covering all input vectors (user, tool, external data).

Penetration test all prompt injection pathways in the system. Prioritise external tool and document ingestion channels.

Build the versioned injection corpus into CI/CD as a pre-release gate. Baseline attack success and sign off the release threshold.

Prioritise jailbreak and adversarial safety testing in pre-deployment validation. Block deployment if prohibited outputs pass filter.

Conduct targeted red team exercises to elicit toxic outputs through jailbreaks and adversarial prompts. Treat bypass as blocking defect.

Test for overconfidence patterns (high-confidence wrong answers, low refusal rate) in pre-deployment validation.

Build a synthetic evaluation dataset of overconfidence-prone scenarios for ongoing regression testing.

Calibrate the groundedness threshold against the hallucination test suite pre-release; sign off the threshold in the validation pack.

Verify a signed attestation and content hash on every dataset shard at ingestion. Reject unsigned or hash-mismatched data before it reaches the training pipeline.

Gate every model promotion on backdoor-trigger probes and a behavioral diff against the approved baseline. Block release on significant regressions or trigger-pattern anomalies.

Attack each candidate model with membership-, attribute-, and inversion-inference harnesses before promotion. Block release when attack advantage exceeds the agreed ceiling.

Configure per-principal budgets and probing-detection rules on the gateway before exposure. Verify enforcement with synthetic attack traffic.

Conduct comprehensive prompt injection red team exercises (direct, indirect, multi-turn) before deployment.

Classify content sources into trust tiers at design; place privileged tools behind a tier requiring user-originated intent or human approval. Sign off the trust-tier map before build.

Re-run injection evals on every template change and periodically against new attack techniques. Manage the spotlighting wrapper under change control.

Use user feedback, reviewer escalations, and monitoring signals to identify and remediate content safety gaps iteratively.

Track accuracy of high-confidence predictions in production. Trigger recalibration when overconfidence rates trend upward.

Require user-facing interfaces to disclose Gen AI limitations and hallucination risk before go-live.

Score every RAG answer for groundedness before release; block, fall back, or escalate responses below the faithfulness threshold.

Sample multiple generations for high-stakes queries and abstain, fall back, or escalate when semantic entropy exceeds the calibrated threshold.

Penetration test the training data pipeline to identify injection points and access control weaknesses.

Scan every ingestion batch with spectral-signature and clustering detectors before training. Quarantine flagged clusters for human review against documented thresholds.

Define the minimum response surface and test it with membership/attribute-inference probes pre-release. Block promotion if any probe recovers raw confidence signals.

Meter inference traffic per principal and flag probing signatures with behavioural analytics. Throttle, step-up, or suspend flagged sessions.

Execute red team tests targeting adverse impact boundary cases and edge population scenarios.

Collect adverse outcome feedback from affected users. Use reports to trigger model updates when adverse impact exceeds threshold.