Audit-grade compliance review ships under multi-layer guardrails

Manual compliance review of vendor and onboarding documents was the bottleneck for new-customer activation. Every traffic spike threatened SLA breach. Reviewer fatigue led to inconsistent flagging — some weeks too strict, some weeks too loose, with no defensible pattern.

• 1Customer activation average of 36+ hours, mostly waiting on compliance
• 2Reviewer fatigue producing inconsistent flag decisions
• 3No machine-readable audit trail — every decision lived in a reviewer's head
• 4Quarterly internal audits surfaced inconsistencies the team could not defend

A single-agent system wrapped in four guardrail layers: an input filter that detects and redacts PII / strips prompt-injection patterns; a versioned policy registry the agent must cite by clause ID for every conclusion; output validators (schema + LLM-as-judge cross-check); and a human-in-the-loop gate on anything scored above a defined risk threshold. Every decision is appended to an immutable audit log.

Documents flow in from the customer's onboarding system. The agent's draft decision (with clause citations) appears in the existing compliance dashboard. Reviewers see the agent's reasoning and either approve, override, or escalate — all logged.

• Onboarding system webhook → agent service
• Compliance dashboard: existing UI, agent draft inline
• Policy registry: in-repo, versioned, clause-IDed
• Audit log: PostgreSQL append-only ledger, queryable by auditors

Every layer of the guardrail stack emits a structured event. Auditors and compliance reviewers see the full chain — input transformations, policy clauses cited, validator passes, human approver (if any), final ruling — in a single dashboard.

• Per-decision trace with all layer events
• Override + escalation rate as leading drift indicators
• Quarterly adversarial eval results published to the audit team
• Replay tooling: any historic decision can be re-run end-to-end

Reviewers no longer do first-pass classification — they review the agent's draft ruling with the cited clauses, confirm or override, and move on. The work that remains is the high-judgement work that genuinely needs human attention.

Business outcomes

Reviewer throughput tripled without adding headcount. Audit findings on AI-assisted decisions: zero across four quarterly reviews. The activation-time reduction also lifted top-of-funnel conversion measurably.

• 01Layered guardrails work. A jailbreak that bypassed the input filter still had to pass the policy-citation requirement, the output validator, and the human gate above threshold. No single failure compromised the system.
• 02The policy registry was the most valuable artefact produced. Even outside the agent context, having policy clauses externalised and versioned changed how the team reviewed decisions.
• 03Adversarial eval cases produced the largest accuracy jumps. Happy-path evals gave false confidence; the cases that mattered were the malformed, ambiguous, and conflicting ones.

The guardrail pattern carries across regulated workflows. The same layered architecture now backs the customer's claims-triage pipeline and is being scoped for transaction-review.

• Claims-triage agent reusing input filter + policy registry
• Transaction-review agent in scoping
• Policy registry promoted to a shared org-wide compliance artefact
• Quarterly eval refresh as a permanent governance ritual

Have a regulated workflow you want to safely automate?

Most regulated AI projects fail at audit, not at build. A scoping session walks through the threat model, policy-registry shape, and approval workflow your auditors will ask for.