HIPAA compliant LLM: what it takes in practice
Short answer: there is no such thing as a HIPAA compliant LLM, only a compliant deployment. The model is a component. What determines compliance is which components see protected health information, whether the vendor will sign a business associate agreement, how long data persists, and whether you can reconstruct who saw what. Those are architecture decisions, made before the build.
The model is not the unit of compliance
Vendors advertise HIPAA-ready models. The phrase is close to meaningless, because compliance attaches to the system and the contracts around it, not to weights.
The questions that actually decide it: does PHI reach this component, is there an agreement covering that, is it retained, and can you prove afterwards who accessed it. A model hosted by a vendor who signs a business associate agreement and contractually excludes your data from training can be part of a compliant deployment. The same model consumed through a consumer product cannot.
The three deployment shapes
Hosted with an agreement. A major provider signs a business associate agreement, retention is configured so prompts are not used for training, and the enterprise tier gives you administrative control. Simplest, and sufficient for many organisations.
Hybrid. Retrieval runs inside your infrastructure and only retrieved passages reach a hosted model. The corpus never leaves your boundary, and what does leave is a bounded, auditable slice. Often the pragmatic middle.
Fully self-hosted. Open-weight models inside your network. Nothing leaves. You now own GPU capacity, model updates, and an inference service, which is a real operational commitment.
Which one you need is a compliance position, not a preference, and it should be settled in week one because everything downstream inherits it.
Draw the PHI boundary explicitly
Write down every component and mark whether PHI crosses it. Ingestion, embedding, vector store, cache, logs, traces, evaluation data, error monitoring.
The ones teams miss are the last four. Traces capture prompts. Logs capture inputs. Evaluation sets are built from real encounters. Error monitoring services receive stack traces containing request bodies. Each is a place PHI leaves the boundary quietly, and each needs either an agreement or redaction.
Retention, and why vector stores complicate it
A vector store holds embeddings derived from PHI. Embeddings are not de-identified: they are a transformation that retains enough of the source to be re-associable, and should be treated as PHI.
That matters for deletion. A retention policy requiring removal after a period has to remove the embedding, not merely the source document. If your ingestion pipeline cannot delete a specific record’s vectors, you do not have a retention policy, you have an intention.
De-identification is not a free pass
Stripping identifiers before sending text to a model is a reasonable control and not a complete one. Clinical narrative frequently re-identifies through context: a rare condition, a specific date, a named facility. Treat de-identification as defence in depth rather than as a boundary.
Audit, and what it must reconstruct
Which user made which request, what was retrieved, what the system returned, and when. Enough to answer a subject access request or an investigation months later.
This is the same requirement that makes the system debuggable, which is a rare case of compliance and engineering wanting the same thing.
Uncertainty is a safety requirement here
The distinguishing constraint in healthcare is not accuracy, it is that the system must surface doubt rather than produce a confident wrong answer.
That means confidence scoring on every generated or extracted field, an escalation path, and a clinician review step before anything is filed. On the clinical orchestrator we built, the records agent carries zero citation hallucinations because answers are assembled from retrieved passages with sources attached, and when retrieval finds nothing supporting, it says so. That is structural, not a prompt instruction. The detail is in the clinical multi-agent API case study.
The takeaway
HIPAA is an architecture constraint, not a checklist applied at the end. Decide the deployment shape, draw the boundary including logs and traces, treat embeddings as PHI for retention, and design for surfaced uncertainty. Decided up front these are ordinary engineering. Retrofitted, they usually mean rebuilding the retrieval layer.
EpochC builds clinical AI for healthcare organisations, including AI medical scribe development, designed around the PHI boundary from week one. Start a project.