HIPAA LLM Compliance Guide

Last month, I accidentally pasted "John K. from ward 4B" into a prompt while testing a triage bot. The model replied with a very confident care summary. I realized, with a cold coffee in my hand, that we had no redaction and no proper logging. I laughed, then panicked, then wrote scripts to scrub logs at 2 a.m. The bug was funny until I thought about auditors.

I want to start practical. The phrase HIPAA LLM compliance captures a lot of fear and a little confusion for teams building healthcare AI. I have worked on production systems at startups and bigger firms. I will explain what matters, what to check, and how to make systems auditable. I write this for CTOs, founders, and engineering leads who need clear next steps.

Why HIPAA Matters for LLMs

LLMs can see and create text that contains protected health information, or PHI. PHI is any data that can identify a patient and relates to health. If your prompts, fine-tuning data, or logs contain PHI, you are in scope for HIPAA. A leaked prompt is not just theory; it is a breach. Regulators and customers will treat it as such.

Think of an LLM like a shared whiteboard in a clinic. Anyone with access can read past notes unless you wipe and log carefully. If you do not control who sees the board, any mistake is visible to the whole clinic. That is the risk with models and logs.

What makes an LLM HIPAA-compliant?

• A compliant setup keeps PHI private across the whole lifecycle. That means secure transport, controlled storage, access controls, and logging that supports audits. It also means business rules, contracts, and documented processes. Technical and operational safeguards must work together.

Can I use ChatGPT or Google Gemini with PHI?

• Most public models are not safe for PHI by default. Some vendors offer business tiers with BAAs and data isolation. You must check the vendor contract, get a signed BAA, and verify that the vendor does not use your data to train public models. Do not assume a public API is safe for PHI.

HIPAA Basics Every CTO Must Know

You need a few core definitions and contracts to move forward.

• PHI: Any health-related data that can identify a person. Names, dates, addresses, billing records, and clinical notes are PHI.

• Covered entities: Providers, health plans, and clearinghouses.

• Business associates: Vendors who handle PHI on behalf of covered entities. LLM vendors can be business associates.

A Business Associate Agreement, or BAA, is not optional if the vendor will access PHI on your behalf. A BAA is like a signed lease for a data room. It specifies who is responsible and how data is protected. Treat it as a service level promise for privacy.

Core safeguards fall into three buckets:

• Administrative controls: Policies, training, and risk assessments.

• Physical controls: Secure data centers and access logs.

• Technical controls: Encryption, authentication, and audit trails.

Do I need a BAA to use an LLM?

• Yes, if the LLM vendor will receive or process PHI. If you keep all PHI inside your environment and the vendor only gets deidentified data, you may not need a BAA. Always confirm with legal.

What is PHI?

• PHI is any information that identifies a patient and is related to health care, treatment, or payment. Names, MRNs, full dates, and precise locations are examples.

How HIPAA Applies to LLM Data Flows

Map your data. PHI can touch many places.

• Client side: Patient names may be entered by users or bots, or live in app state.

• API calls: The prompt you send can include PHI.

• Model endpoints: The model processes inputs and produces outputs that might be stored.

• Logs and telemetry: Inputs, outputs, and traces often end up in log files.

• Backups: Archives or snapshots may contain PHI.

Distinguish three cases:

• Training data: Where you may fine-tune or build models with data.

• Inference inputs: What users send to get a response.

• Telemetry logs: Which record what happened.

Every place PHI appears needs controls, not just the model store. Visualize the flow like plumbing. A leak at any joint floods downstream systems. That is why you must secure each handoff.

Can an LLM store PHI?

• Models can memorize data. If PHI appears in training or prompt logs, models may reproduce it. So yes, an LLM can effectively store PHI. Control training inputs and avoid exposing PHI to public models.

How do you log LLM activity for audits?

• Keep structured logs with timestamps, user IDs, prompts, model versions, and outputs. Redact or mask PHI where full content is not needed. Store logs in immutable archives and correlate them with access controls.

Minimum Technical Controls for HIPAA LLM Compliance

Technical controls are non-negotiable for production systems.

• Encryption at rest and in transit: Use keys you control when possible. If the vendor manages keys, confirm key management policies.

• Access control and least privilege practices: Limit who can call model endpoints and who can read logs.

• Strong authentication, role-based access, and session controls: Tie actions to identities for auditing.

Think of encryption like putting PHI in a locked safe at rest and in transit, not just a drawer. Locks are only useful if you control the keys and track who opens them.

Is encryption enough for HIPAA compliance?

• No. Encryption is necessary, but not sufficient. You also need access control, logging, training, policies, and BAAs. Compliance is a system property, not a single control.

Operational Controls and Policies

Technical controls must be backed by policy and operations.

• Data minimization: Only send what is necessary. Use prompt templates to avoid free text entries.

• Redaction: Always strip or mask PHI before sending to non-approved services.

• Training and access rules: People who craft prompts or review outputs should have training and limited access.

Think of prompt templates like a form doctors fill out to avoid free text mistakes. A good form removes ambiguity and makes audit trails simple.

What are best practices for using LLMs in healthcare?

• Start with mapping data flows. Use templates and redaction. Get BAAs. Apply strict access controls. Log everything in a structured way. Pilot in a controlled environment before wide rollout.

Vendor Evaluation Checklist for HIPAA LLMs

When you pick a vendor, check these items.

• Signed BAA: Ensure it covers logs, backups, and subcontractors.

• Data use policy: Will the vendor train models on customer data? How long is data retained?

• Security posture: Look for SOC2 reports, pen test results, encryption standards, and data separation.

Choosing an LLM vendor is like choosing a hospital lab. You check credentials, processes, and who else touches samples. You also ask if they will reuse the samples for training others.

Can I use ChatGPT or Google Gemini with PHI?

• Only if you have a signed BAA and the vendor offers a product tier that isolates customer data and prevents training on your PHI. Default consumer tiers are unsafe for PHI.

What makes an LLM HIPAA-compliant?

• A combination of contractual, technical, and operational controls. This includes BAAs, data isolation, encryption, access control, logging, and documented processes.

Training and Fine-Tuning on PHI, Yes or No

Fine-tuning on PHI is high risk.

• If you train on PHI, you must isolate that process and lock down data access. Legal review is required.

• Prefer alternatives, like prompt engineering, private models with strict access, or synthetic data.

• If you do train, document the intent, use encryption, and monitor for model memorization.

Think of fine-tuning on PHI like teaching a student with real patient charts. You must control what they remember and document why the student needs the charts.

Can I train an LLM on PHI?

• Technically yes, but it is risky. Only do so with legal sign-off, strict controls, and monitoring.

Is training your own full LLM on PHI allowed?

• It can be allowed, but it requires the same safeguards and a high bar for documentation, isolation, and monitoring. Many teams choose private models or synthetic data instead.

Observability, Logging, and Audit Readiness

Good logs make or break audits.

• Collect structured logs of inputs, outputs, user IDs, timestamps, and model versions.

• Mask or redact PHI in copies where full retention is not needed.

• Use an immutable store for audit logs and define retention policies.

Logs are like CCTV for data. You need clear cameras, synchronized timestamps, and a secure vault for recordings.

How do you log LLM activity for audits?

• Log the prompt ID, prompt version, user ID, time, model version, and the response. Keep an unalterable audit trail and redact PHI when you do not need full content.

What should be in LLM logs for HIPAA?

• At minimum, user identity, timestamps, model and prompt version, input hash, and output hash. Include full content only when needed and store that in a protected, access-controlled way.

How LaikaTest Logs Can Support HIPAA Audits

LaikaTest provides structured, tamper-evident logs that link prompts, responses, user context, and model version. This is crucial during an audit. You can show exactly which prompt variant ran, who triggered it, and what the outcome was.

• Filter logs to show PHI access events, time windows, and user actions.

• Produce incident timelines quickly, with proofs of redaction or manual review steps.

• Show experiments and who approved them. That reduces the time to respond in an audit.

Think of LaikaTest as a certified ledger for each LLM interaction. When an auditor asks for the chain of custody, you can produce it without digging through ad hoc log files.

How do you log LLM activity for audits?

• Use a tool that ties prompts to versions, model calls, and user actions. LaikaTest does this in one line of observability, so you can see which prompt version was used and why.

Can logs help during a HIPAA investigation?

• Yes. Immutable, structured logs let you scope incidents, find affected PHI, and document remediation steps. That helps meet reporting timelines.

Incident Response and Breach Notification

Have a runbook that is ready before you need it.

• Containment steps: Stop writes, revoke keys, and isolate affected services.

• Forensic logging: Capture detailed timelines and preserve evidence.

• Notification: Follow HIPAA breach rules and timelines for reporting.

Incident response is like a fire drill for data. Practice it often so people know what to do and do not panic.

What do I do if an LLM leaks PHI?

1. Contain the system and stop further leaks.

2. Use logs to identify exposed PHI and affected users.

3. Notify your legal and compliance teams.

4. Follow HIPAA breach notification rules and timelines.

5. Remediate root causes and update controls.

What are breach reporting rules under HIPAA?

• HIPAA has specific timelines and thresholds. Small breaches may need internal reporting, while large breaches require OCR notifications and public notices. Consult legal and compliance to determine requirements.

Practical Implementation Checklist

Think of this as a pre-flight list before each LLM launch in production.

1. Map data flows and classify PHI touchpoints.

2. Obtain BAAs for vendors that handle PHI.

3. Apply encryption at rest and in transit and manage keys.

4. Implement IAM and least privilege for APIs and logs.

5. Set up structured logging with retention and immutability.

6. Train staff on prompts, redaction, and incident response.

7. Run risk assessments and document model use cases.

8. Use LaikaTest to capture immutable traces and experiment safely.

What are best practices for deploying LLMs in healthcare?

• Pilot with strict guardrails. Use templates and redaction. Limit access. Log everything. Get BAAs. Use LaikaTest for traceability.

How do I prepare for a HIPAA audit with LLMs?

• Gather BAAs, maps of data flows, access logs, and your incident playbooks. Use tools that can export immutable logs and show versioned prompts.

FAQ for Founders and CTOs

This is a quick triage for urgent questions.

Can I use ChatGPT or Google Gemini with PHI?

• Only with a proper business contract and BAA for data isolation. Consumer products are not safe for PHI.

Do I need a BAA to use an LLM?

• Yes, if the vendor processes PHI. If you never expose PHI to the vendor, consult legal.

Is encryption enough for HIPAA compliance?

• No. Encryption helps, but you also need access control, logging, BAAs, and operational controls.

This week, do three things. Map where PHI flows, request BAAs from any vendor that may see PHI, and turn on structured logging with immutable retention.

Conclusion with LaikaTest

HIPAA LLM compliance is a collection of practices, not a single checkbox. Map your PHI flows, get BAAs, apply encryption and access controls, and put strong logging in place. Use prompt templates and redaction to reduce risk. Train staff and run incident drills. For logging and traceability, consider LaikaTest as a practical tool. It gives you structured, tamper-evident logs, ties prompts to versions, and lets you run safe experiments with prompt A/B testing. LaikaTest helps you show auditors exactly what happened and supports the evaluation feedback loop you need to improve models without losing auditability.

For policy-level work and governance, see the Enterprise AI Quality & Governance pillar page. For deeper logging and tracing best practices, see the LLM Observability & Tracing pillar page.

Take these next steps now. Map PHI flows. Get BAAs. Apply encryption and IAM. Use LaikaTest to capture immutable traces for audits. That combination will make your LLM systems safer and make compliance audits much easier.