HIPAA-Compliant AI Tools: What Healthcare Practices Should Actually Look For

Your Practice Needs Compliant AI. Here's How to Evaluate It.

If you are reading this, you have probably already decided that your practice needs AI tools that work with patient data. You have seen what AI can do for documentation, prior authorizations, coding, and clinical correspondence. And you know that consumer ChatGPT is not HIPAA compliant.

The market has responded. There are now dozens of products marketing themselves as HIPAA-compliant AI for healthcare. Some are genuinely well built. Some are a BAA stapled to a third-party API. The challenge for a practice administrator or physician-owner evaluating these tools is that they all sound the same on the surface — and the differences that matter are architectural, not cosmetic.

Here are the four questions that separate compliant AI tools that actually protect your practice from ones that just check the compliance box.

1. Where Does Your Data Go, and Who Touches It?

This is the most important question, and the one most product marketing obscures.

When you type a patient's name and diagnosis into an AI tool, that data has to be processed somewhere. The question is where, and how many entities handle it along the way.

Most HIPAA-compliant AI tools marketed to healthcare practices are middleware. The product you interact with is a front-end application — a chat interface with a BAA. Behind it, your data is routed to a third-party large language model provider like OpenAI, Anthropic, or Google for processing. The middleware vendor has a BAA with your practice. The middleware vendor also has a separate agreement with the LLM provider.

Your practice's compliance position now depends on a chain: your practice, the middleware vendor, and the underlying LLM provider. Each link has its own BAA scope, its own data handling terms, and its own potential for change. If the LLM provider modifies its terms or data practices, your compliance posture shifts — even if your direct vendor has not changed anything.

For a practice with a compliance officer who can evaluate and monitor that chain, this may be acceptable. For the majority of independent practices, understanding and tracking two sets of vendor terms is not realistic.

2. What Does the BAA Actually Cover?

A BAA is a legal agreement, and like all legal agreements, scope matters. The presence of a BAA does not mean every feature of the tool is covered.

We documented this in detail with Microsoft Copilot, where web search queries are explicitly excluded from HIPAA coverage under the BAA. The same principle applies to any tool: file uploads may be handled differently from chat queries. Certain model features or integrations may fall outside the BAA's scope. If the tool connects to external services for retrieval or search, those data pathways may not be covered.

The practice should ask the vendor directly: which specific data pathways and features are covered by the BAA, and which are not? If the vendor cannot answer that question clearly, that is the answer.

3. How Is PHI Actually Handled?

Not all compliant AI tools handle patient data the same way. There are three fundamentally different architectures, and the tradeoffs between them matter.

De-identification before processing. Some tools strip protected health information from your input before sending it to a third-party AI model, then re-insert the identifiers into the response. This means the LLM never sees actual PHI — it processes sanitized text. The appeal is clear: if the AI model never sees PHI, the HIPAA exposure to the LLM provider is minimized.

The limitation is equally clear. De-identification is imperfect. Clinical text is complex, and automated stripping can miss identifiers or remove context that the AI needs to generate an accurate response. A prior authorization letter that requires the patient's specific diagnosis history, medication list, and insurance details cannot be drafted from de-identified text — the AI needs the real information to do useful work. For simple summarization tasks, de-identification may be adequate. For the documentation-heavy workflows that consume the most practice time, it creates a ceiling on what the AI can actually do.

Third-party processing with BAA coverage. The middleware model described above. PHI is sent to a third-party LLM with BAA coverage through the vendor chain. The AI processes the full, identifiable data. This enables the full range of clinical and administrative use cases. The tradeoff is the vendor chain dependency and the trust placed in the LLM provider's infrastructure and data handling.

Processing on infrastructure you control. Private AI runs the models on servers within an environment the practice controls. PHI never leaves that environment. There is no vendor chain for data processing, no third-party LLM provider handling patient information, and no BAA carve-outs to track. The AI processes the full, identifiable data — enabling the same use cases as the middleware model — but entirely within the practice's security perimeter.

4. Who Maintains Compliance — You or the Provider?

Some tools require the practice to configure and maintain the compliance posture. Enterprise platforms like Microsoft Copilot require E5 licensing, DLP policies, sensitivity labels, access controls, web search disabling, and ongoing monitoring. The tool enables compliance. The practice creates it.

Other tools handle compliance within the platform. The practice logs in and uses the tool. Encryption, access controls, audit logging, and data isolation are built in and maintained by the provider.

For independent practices without dedicated IT staff, this distinction determines whether a tool is usable in practice, not just in theory. A tool that requires an enterprise compliance program is not a realistic option for a 12-provider group — regardless of how capable its AI is.

Which Approach Fits Your Practice?

There is no single right answer. The right approach depends on the practice's size, resources, and tolerance for vendor dependency.

Large health systems with dedicated IT security and compliance teams can manage enterprise platform AI or evaluate middleware vendor chains. They have the staff to configure controls, review BAA terms, and monitor ongoing compliance across multiple vendors.

Tech-forward practices comfortable with SaaS evaluation may find the middleware tools adequate, especially for simpler use cases where de-identification or vendor chain dependency is an acceptable tradeoff.

Independent physician-owned practices — groups of 5 to 30 providers without full-time IT staff — benefit most from the approach that requires the least ongoing compliance management. Private AI on infrastructure the practice controls offers a single BAA, no vendor chain, no configuration burden, and a defensible answer to every compliance question about where patient data is processed.

The standard for evaluating any HIPAA-compliant AI tool is not whether it has a BAA. It is whether the practice can explain, clearly and completely, where patient data goes when someone on staff uses the tool. If the answer involves multiple vendors, multiple BAAs, and multiple sets of terms that the practice needs to monitor — the practice should consider whether a simpler architecture exists.

Learn how Metrovolo deploys HIPAA-compliant private AI for healthcare practices, or book a demo to see how it works for your practice.