Residency enforced by
topology, not convention

The claim

Three artifacts, thirty seconds

Everything in this build exists to make these three moments real. Two of them are failures.

The problem

Why regulated AI pilots stall

Architecture

What was deployed

Two lanes with genuinely different trust boundaries. The gateway sits in a public subnet and can reach both. The model host sits in a private subnet whose route table is empty. That asymmetry is the control.

Enforcement

How the decision is made

One hook, one policy file, one route table. Classification completes before any provider is contacted, and every branch — allow and deny alike — writes an audit record.

Two decisions worth defending

• Detection is local, not an LLM call. Sending text to a model to ask whether it contains PHI means the PHI has already left the trust boundary. Presidio runs in-process, baked into the gateway image, with no network dependency — so it cannot fail open because a detection service was unreachable. Measured overhead: about 7 ms.
• The caller's claim is evidence, not authority. The x-data-class header is recorded, never obeyed. A developer who mislabels a request — carelessly or in good faith — does not thereby widen the egress boundary.

Policy is data, not code

A privacy officer who cannot read Python can read this, diff it across versions, and see exactly when it changed:

lanes:
  in_tenancy:  { phi_permitted: true,  egress: none }
  frontier:    { phi_permitted: false, egress: internet }
on_violation: deny

Changing one word opens PHI egress with no code change and no rebuild. That is the strength of the design and equally its risk — which is why the enforcement test asserts PHI executed outside the tenancy: 0 and belongs in a CI gate blocking merges to that file.

Evidence

Captured from the running system

Network isolation, verified live

$ ./03-seal-subnet.sh seal
  Private subnet is now: SEALED — no internet route

$ ssh -J gateway model 'curl -m 10 https://api.<provider>.com'
  curl: (7) Network is unreachable
  BLOCKED

$ python scenarios.py 2          # in-tenancy inference, same moment
  200 ALLOW  served by: hosted_vllm/meta-llama/Llama-3.1-8B-Instruct

The compliance answer

PHI executed outside the tenancy:      0
Caller data-class claims overridden:   1
Reason codes:  { OK: 3, PHI_LANE_VIOLATION: 2 }

Findings

What broke, and what it means

Naming your own gaps is most of what separates a credible pattern from an oversold prototype. These were found by testing the classifier separately from the pipeline — which is the only reason they were found at all.

Platform notes

Scope boundaries

• Not HIPAA Safe Harbor. Safe Harbor enumerates 18 identifier classes; this detects a subset. Imagery, biometrics and device identifiers are out of scope entirely.
• The de-identification lane is a stub. deid_applied is always false. The architecture has the seam; the implementation does not fill it.
• No HA. Single gateway instance, single model host. This proves a control surface, not an SLA.
• Enforcement, not certification. This enforces where inference runs and records what happened. BAA coverage, clinical validation and the de-identification method remain the customer's responsibility.

Phase two

The same control, native to the platform

The proof above uses a purpose-built gateway. The next question was whether the same control survives being implemented the way a real platform would consume it — as a registered guardrail inside LiteLLM, rather than a bolted-on callback.

It does. Classification runs in-process before egress, PHI-bearing requests to non-permitted models are denied, and the platform records the intervention itself. Steady-state overhead is under 6 ms.

What the platform records, and what it doesn't

The platform stores guardrail_response: "REDACTED_BY_LITELM", which is a sound default — a guardrail's response could echo the very content it exists to protect. The consequence is a clean split: the platform supplies the execution record, the guard supplies the compliance reasoning, and they join on request ID.

Config is the right home for a compliance control

• A config-defined guardrail cannot be disabled from the console. No toggle exists in the UI.
• It cannot be shadowed. Creating a same-named guardrail through the console does not override or replace it.
• The policy file is invisible to the UI. It stays a version-controlled artifact a privacy officer can review, outside the console's reach.

That is specific to guardrails, and worth stating precisely rather than generalising — store_prompts_in_spend_logs behaves the opposite way, and can be set in the console to override config.

Finding

The control plane is in scope

The gateway's database holds keys, teams, spend and model configuration. It can also hold request and response content. The default is safe — but the setting that changes it is a console click, and the content lands somewhere the documentation doesn't name.

The recommendation: treat the control-plane database as in scope for regulated data regardless of configuration, and audit it column-agnostically rather than against the documented field.

The pattern

The dangerous failure is the silent one

Five independent failures surfaced across this work. None of them announced itself. Every one looked like success from the outside.

Generalization

The same shape, a different policy file

PHI is one instance of a pattern that recurs across every regulated conversation: classify at the door, enforce with topology, record the decision.