You open your inbox and find a compliance questionnaire asking exactly where patient prompts are stored and whether any model trainer has seen them.
If you are choosing an LLM router for regulated industries in October 2026, you need a gateway that terminates PHI at the edge, keeps prompts out of training sets, and routes each request through a region you control. For most healthcare and regulated teams, the real choice is between a purpose-built gateway with a signed BAA and a self-hosted router you operate yourself. Managed model marketplaces without a BAA are not a compliant starting point.
Key takeaways
- An LLM router is the traffic layer between your app and the model, not the model itself. Decagon's glossary describes it as the component that decides which model handles each request.
- HIPAA compliance requires a signed BAA; it is not a product certification. On CloudSitus, BAA coverage exists only on the Azure routes.
- If the gateway logs payloads, you have a compliance problem before the first request. Look for stateless, no-payload-logging designs with an audit trail on every call.
- GDPR and nFADP add residency and transfer obligations that a BAA alone does not satisfy; ask your counsel before routing EU or Swiss patient data.
- Build it yourself only if you have the team to maintain auth, logging, and region pinning; otherwise buy a gateway that already publishes those controls.
What is an LLM router, and does a regulated team need one?
An LLM router receives your application's request and decides which model endpoint handles it, rather than your app calling a model directly. Decagon defines it as the layer that sits between your code and the model and makes routing decisions per request. For regulated teams, that position matters because the router is where you can inspect, redact, or block PHI before it ever reaches a model provider.
You need one when your application sends sensitive text to third-party models and you cannot trust the model provider's default handling. The router becomes your enforcement point: it can strip identifiers, enforce region residency, and write an immutable audit record. If you are only calling a single model from a single region with no sensitive data, you probably do not need a router yet.
AI gateway healthcare: does HIPAA BAA or GDPR/nFADP apply?
If you are routing healthcare data, HIPAA and GDPR/nFADP are different regimes and neither is satisfied by the other automatically. A HIPAA BAA is a contract between a covered entity and a business associate; it is not a product certification, and no gateway should be described as "HIPAA-certified." On CloudSitus, BAA coverage is available only on the Azure routes.
GDPR and the Swiss nFADP concern data residency, transfers, and subject rights. If your patients are in the EU or Switzerland, routing their PHI to a model in another region may create a transfer problem even if you have a BAA. Ask your counsel before assuming a BAA covers cross-border healthcare traffic.
OpenAI compatible API gateway BAA: what does the BAA actually cover?
An OpenAI-compatible API gateway lets your application keep using the OpenAI SDK while the gateway forwards requests to a different backend. The BAA, where available, covers the gateway operator's handling of PHI as a business associate; it does not automatically extend to every model the gateway can reach.
On CloudSitus, the mechanism is stateless per-request routing: the gateway inspects the request, applies the PHI gate, forwards to the selected model in the chosen region, and returns the response. Payloads are not logged, and the service does not train on your data. A curl call looks like this:
curl https://cloudsitus.com/api/v1/chat/completions \
-H "Authorization: Bearer $PHI_TOKEN" \
-H "Phi-Region: us-east-1" \
-d '{"model":"gpt-4o","messages":[{"role":"user","content":"..."}]}'
Response headers include a request ID for audit correlation, but the response body is not retained by the gateway. Ask your counsel whether a specific BAA covers the model provider downstream of the gateway.
LLM routing compliance: what goes in the audit trail?
Every call through a compliant router should produce an audit record: request timestamp, source application, model endpoint, region, routing decision, and a redacted or hashed identifier for the user. The payload itself should not be stored. If your gateway cannot produce that trail, an auditor will treat the model provider as an unvetted data processor.
Compliance also means the router cannot silently retrain on your traffic. No-training clauses and zero-retention policies belong in the BAA or the service agreement, not just in marketing copy. Inworld positions its inference offering around low latency, publishing a claim of nearly 8× faster time to first token than most other providers as of March 2026; speed does not replace compliance, but it shows where vendors compete.
PHI data routing LLM: where does the gate live?
The PHI gate should live at the router, not inside the model and not only at your application layer. A per-request PHI gate inspects each incoming payload, applies rules (block, redact, allow), and then decides whether to forward, transform, or reject the request. Because the gate is stateless, it does not keep a copy of the message after the response is sent.
This matters for regulated industries because the application developer can forget to scrub a field, but the router cannot if it is configured correctly. The trade-off is latency: inspection adds milliseconds. For most healthcare workloads, that cost is cheaper than a breach notification.
OpenRouter alternatives enterprise: how do the options compare?
Enterprise buyers often start with OpenRouter because it aggregates many models behind one OpenAI-compatible endpoint. The advantage is convenience: one key, many models. The disadvantage for regulated industries is that convenience does not come with a BAA or region guarantees you can sign. GetMaxim's 2026 roundup lists multiple router solutions, but the cited sources do not publish OpenRouter's BAA or residency terms, so those cells are marked not published.
Here is how the options compare on the dimensions that matter for regulated work.
| Dimension | CloudSitus | Inworld Inference | Self-hosted open-source gateway |
|---|---|---|---|
| Primary role | Per-request PHI router with BAA | Managed inference gateway | Router you operate |
| OpenAI-compatible API | Yes | Not published in cited sources | Depends on the project |
| BAA coverage | Azure routes only | Not published in cited sources | You arrange with your infra provider |
| Data residency | Customer-selected region | Not published in cited sources | Wherever you host |
| Payload logging | No payload logging | Not published in cited sources | You control it |
| Audit trail on every call | Yes | Not published in cited sources | You implement it |
| Stateless per-request PHI gate | Yes | Not published in cited sources | You implement it |
| Published performance claim | Not published | Nearly 8× faster TTFT (Mar 2026) | Depends on your hardware |
The honest trade-off: managed gateways move faster but you must trust their BAA scope and logging claims. Self-hosted routers give you full control but you become the compliance officer, the on-call engineer, and the security reviewer all at once.
AWS Bedrock alternatives healthcare: should you build or buy?
If you are already on AWS, Bedrock is the obvious default, and its advantage is that your data can stay inside your existing AWS accounts and VPCs. The cited sources do not publish Bedrock's BAA terms or PHI routing specifics, so those details are marked not published; check AWS directly before signing.
The build-or-buy question comes down to who owns the audit trail. Buying a gateway means the vendor owns the PHI gate and the BAA, but you are still responsible for configuring it correctly. Building means you own every header, every log line, and every region setting, but you also own the outage when the routing layer fails at 2 a.m. For teams without a dedicated platform engineer, buying is usually the lower-risk choice.
If you are evaluating gateways for regulated workloads, start by asking for the BAA, the region list, and the logging policy before you ask about model selection. CloudSitus publishes its per-request PHI routing and audit-trail design for teams that want those controls out of the box.
FAQ
What is an LLM router and why does it matter for regulated industries?
It is the layer between your app and the model that decides where each request goes. For regulated industries, that position lets you enforce PHI gating, region residency, and audit trails before sensitive data reaches a model provider.
Does a BAA make an AI gateway HIPAA compliant?
No. A BAA is a contract, not a certification, and no gateway should be called "HIPAA-certified." On CloudSitus, BAA coverage is limited to the Azure routes; ask your counsel whether a specific agreement covers your use case.
Can I route PHI to an OpenAI-compatible gateway?
Only if the gateway operator has signed a BAA covering that route and your counsel approves the data flow. Compatibility with the OpenAI API does not imply HIPAA coverage.
Should I build my own LLM router or use a managed gateway?
Build if you have a platform team and need full control over logging and residency. Buy if you want a pre-signed BAA, no-payload-logging defaults, and an audit trail without operating the infrastructure yourself.
What should go in the audit trail for every LLM call?
Request time, source application, model endpoint, region, routing decision, and a redacted user identifier. The message payload itself should not be retained by the gateway.