AI has moved into healthcare fast. Faster than most compliance programs have kept up. Covered entities and their business associates are applying AI across a growing range of use cases, not always knowingly. The compliance exposure follows.
The Different Ways AI Is Entering HIPAA-Regulated Organizations
AI is not arriving as a single, obvious change. It is appearing in different forms, each carrying different risks and varying levels of observability. That combination is making HIPAA compliance materially harder to maintain.
Part of what makes AI hard to govern is that it turns an existing problem into a larger one. Healthcare organizations already had an integration layer: APIs, webhooks, and SaaS-to-SaaS connections moving ePHI across system boundaries faster than compliance teams could track.
AI governance is not just a question of vendor contracts. If ePHI was already moving across ungoverned boundaries, AI is an issue multiplier.
Examples of the Types of AI in Healthcare
Much of the acceleration of AI usage within healthcare settings has been driven by the emergence of large language models, which have made sophisticated AI capabilities across clinical and administrative functions accessible.
| AI Type | What It Does in Healthcare |
|---|---|
| Rule-Based Systems | Applies if-then logic to support clinical decisions. |
| Machine Learning | Uses historical data to identify patterns and guide decisions without explicit programming. |
| Deep Learning | Neural networks that process unstructured data including imaging, pathology, and clinical notes. |
| Natural Language Processing (NLP) | Interprets and generates human language from clinical text. |
| Robotic Process Automation (RPA) | Automates repetitive rule-based administrative tasks. |
| Predictive Analytics | Forecasts patient outcomes and resource requirements using real-time data. |
| AI Agents | Interacts with patients or staff autonomously, often calling multiple systems to complete a goal. |
How AI Capability Enters HIPAA-Regulated Organizations
How each AI capability is used, where it is hosted, what it is connected to, how it utilizes data (such as global model training) and who governs it determines the compliance exposure it creates.
| Route | Description | Governability |
|---|---|---|
| Dedicated AI Products | Products knowingly procured for a specific function. Ambient scribes, scheduling bots, clinical decision support, prior authorization platforms. | High. Procurement decision was made. Compliance is typically part of the process. |
| Existing Products With AI Features | AI activated inside platforms already in use, through routine product updates. ERP, CRM drafting tools, embedded copilots, AI-assisted billing. | Medium. Existing BAA agreement may not cover new AI data flows. Sometimes an organization are not aware of what data is accessed and if up-stream AI processors are utilized. |
| Connected Apps and Marketplace Additions | Third-party AI tools and plugins added to existing platforms through app marketplaces or direct integrations. | Medium. Installed by users or teams without IT or compliance involvement. Parent platform BAA almost certainly does not cover third-party additions. |
| AI Agents | Autonomous systems given a goal. Call multiple systems, make decisions, and execute actions sometimes without human oversight at each step. | Low. Actions occur at machine speed across multiple systems. Each system boundary is a potential compliance gap adding complexity. |
| Shadow AI | Consumer AI tools used by employees without organizational approval, monitoring, or BAA coverage. ChatGPT, Claude, Google Gemini used to draft notes, summarize records, or process billing data. | Very Low. No system alert. No log entry. No audit trail. Exposure can be silent. |
Dedicated AI Products
These are applications or extensions that HIPAA-regulated organizations knowingly procure. Some replace or augment existing workflows with a better version of the same thing: ambient scribes, documentation tools, scheduling bots. Others represent genuinely new capabilities: real-time risk scoring, predictive readmission models, AI-driven prior authorization.
These tools are visible. Someone made a decision to buy them. The compliance path is traceable and HIPAA requirements are typically part of the procurement process. Understanding what systems and data they connect to and how is key here.
Existing SaaS Products With In-built AI Features
This is where the observability, therefore, governability can drop slightly. AI is being activated inside cloud platforms that organizations have used for years, often through routine product updates and new features. Think about CRMs with AI-powered drafting for messages to clients or chatbots.
Three questions matter before enabling any of these features.
-
What data can the AI access and at what level of privilege?
-
What model is actually powering the feature and where does it live? Many SaaS vendors are not running their own foundation models. They are calling third-party models via API.
-
What is covered by the existing BAA agreement, and what are the terms and conditions of this feature?
Connected Apps and Marketplace Additions
Third-party AI tools added to existing platforms through app marketplaces or direct integrations represent a significant compliance gap. Be they, a plugin added to Microsoft 365 or a claims tool from the Salesforce AppExchange. Someone made a decision to install each of these. That decision may have never gone through procurement or compliance.
The parent platform BAA often does not cover third-party additions. Each connected app touching ePHI is a potential business associate requiring its own agreement under the HIPAA Security Rule and worth assessing how they handle ePHI within and outside the parent platform
AI Agents
An AI agent is different in kind, not just degree. You give an agent a goal and it decides what steps to take. This can include calls to other systems, retrieving data, making decisions, and executing, often across multiple platforms, sometimes without any human oversight.
Where an embedded AI feature is passive, it assists a user who initiates a task. An AI agent can be semi-autonomous or autonomous. The distinction is important because the risk profiles are different. Embedded AI features expand ePHI access within a tool you already govern and have hopefully secured.
Agents can potentially create autonomous ePHI flows across system boundaries. Agentic systems are already in production within healthcare settings, and each system call may constitute a separate business associate relationship. Each boundary it crosses is a potential compliance gap.
There is a secondary consideration, and that is the security vulnerability. Where hackers find vulnerabilities within the chain of systems. This is both a general security risk and a compliance one and was clearly illustrated by the ServiceNow vulnerability tracked as CVE-2025-12420 and codenamed BodySnatcher. The vulnerability allowed for an unauthenticated attacker to impersonate an administrator and execute an AI agent to override security controls and create backdoor accounts with full privileges, potentially granting access to sensitive customer information.
Read more: ServiceNow BodySnatcher Vulnerability Exposes Agentic AI Security Gap
Shadow AI: AI Tools Used Without Authorization
We have covered new and known AI products, and AI features embedded into SaaS stacks, but there is a type of AI that has the lowest observability and therefore governability, that is, shadow AI.
Shadow AI is the use of AI tools without the approval, monitoring, or involvement of the organization. A common scenario could be a clinician drafting notes in ChatGPT or an administrator pasting billing records into a browser-based AI tool.
Consumer versions of tools like ChatGPT, Claude, and Google Gemini are not HIPAA-compliant by default. Many are marketed as HIPAA-eligible, which is not the same thing. Eligible means the vendor can enter into a BAA under specific enterprise configurations. Without that configuration and a signed BAA, using those tools with ePHI is an unauthorized disclosure.
Shadow AI, by its nature, is hard to govern because you don't know it is happening. From a risk averse perspective, the best approach is to assume to some degree it is.
The HIPAA Risks AI Creates for Covered Entities and Business Associates
Each of those entry points creates specific compliance exposure under the existing HIPAA Security Rule. But it is vital to note that the regulatory bar is also moving. OCR's proposed Security Rule update, issued December 27, 2024, addresses AI systems for the first time, requiring them to appear in a mandatory technology asset inventory. AI is a known risk vector, and regulations are changing to adapt.
Under the NPRM, Encryption Becomes a Required Safeguard
The new rule proposes to remove the distinction between required and addressable safeguards, making encryption of ePHI a required technical safeguard. Under the current rule, organizations can implement an equivalent measure if encryption is not reasonable and appropriate.
Both changes are proposed, not final. But the risks below exist under the current rule. The proposed rule would make them significantly harder to avoid addressing.
Business Associate Agreements That Do Not Cover AI Use
Every vendor that creates, receives, maintains, or transmits ePHI on behalf of a covered entity must have a valid BAA in place that outlines permissible data use and safeguards 45 CFR 164.308(b)(2).
A BAA signed before a vendor delivered AI features may not cover those features. For AI agents that call multiple systems in a single workflow, each third party receiving ePHI in that chain may need to be assessed as a business associate requiring its own agreement.
AI Training Data That Contains ePHI
When ePHI is used to train or fine-tune a model, it becomes subject to the HIPAA Security Rule. HHS has directed that ePHI in AI training data, prediction models, and algorithm data maintained by a regulated entity falls within the scope of HIPAA Security Rule requirements.
The risk is compounded by shadow AI. If an employee pastes PHI into a consumer AI prompt, that data may be used to improve the vendor's underlying model unless specific enterprise terms explicitly prohibit it. This likely violates existing HIPAA standards. The NPRM deals specifically with this:
"...The Department believes that the adoption of the cybersecurity best practices proposed in this NPRM is an important first step to ensuring that AI tools are deployed by regulated entities in a manner that protects the confidentiality, integrity, and availability of ePHI."
Third-Party Breach Exposure Through AI Vendor Infrastructure
Every AI vendor in the stack is a potential breach vector. Under the HIPAA Security Rule, covered entities must obtain satisfactory assurances that business associates will appropriately safeguard ePHI.
That obligation extends to every vendor handling ePHI, including SaaS platforms that have AI features. According to analysis of OCR breach portal data cited by the American Hospital Association, the number of individuals impacted by healthcare data breaches increased from 27 million in 2020 to 259 million in 2024, with most breaches resulting from hacking incidents targeting third-party vendors.
The proposed Security Rule NPRM would require organizations to maintain a technology asset inventory mapping all systems that touch ePHI. AI vendors that are not on that map are ungoverned by definition.
The Minimum Necessary Standard
The HIPAA Privacy Rule's minimum necessary standard requires that PHI accessed or disclosed is limited to what is strictly necessary for the stated purpose.
AI models often seek comprehensive datasets to optimize performance, which creates tension with that standard. A model trained on full patient records to improve a single workflow may be accessing far more PHI than the task requires.
The Regulatory Outlook: Two Proposals Worth Watching
The current HIPAA Security Rule remains in effect. But two proposed frameworks signal where regulators are headed. Organizations that wait for finalization will be behind from the start.
The HIPAA Security Rule NPRM
On December 27, 2024, OCR proposed the first update to the HIPAA Security Rule in 20 years. For the first time, AI systems are explicitly brought within the rule's scope. The proposals signal that OCR no longer views AI as a separate governance question. It is a Security Rule question. AI software touching ePHI must appear in a mandatory technology asset inventory; encryption becomes a required technical safeguard with no alternative; and a network map of all ePHI flows must be maintained annually.
This is proposed, not final. The HHS OCR fact sheet is the authoritative summary.
HISAA
The Health Infrastructure Security and Accountability Act, introduced September 26, 2024, would sit alongside HIPAA, replacing flexible self-assessed cybersecurity standards with mandatory, independently verified ones. Executives would certify compliance annually, with false certifications potentially resulting in fines of up to $1 million and criminal charges.
HISAA is a proposed bill, not law. Together with the NPRM, it points in one direction: mandatory standards, independent verification, and executive accountability.
Defensive Strategies for Managing HIPAA AI Risk
Build Your AI Inventory Across All Entry Points
You cannot govern what you cannot see. Start by mapping AI across all three entry points: purpose-built tools procured by the organization, AI features or integrations activated inside existing SaaS platforms, and the consumer tools employees are using with and without authorization. Under the proposed Security Rule update, AI software touching ePHI must appear in a mandatory technology asset inventory.
Reduce ePHI Exposure at the Data Layer
The most durable defense is reducing how much real ePHI any AI system ever touches and doing it upstream. Under HIPAA's de-identification standards, properly de-identified data falls outside the definition of PHI entirely. Under the proposed NPRM, encryption would become a required technical safeguard.
Encrypting or tokenizing ePHI before it is transmitted to SaaS means any connected AI system receives only the encrypted or tokenized values,
Audit Every BAA for AI Scope, Not Just Existence
A BAA that predates a vendor's AI feature activation may not cover that feature. Review every BAA against current product capabilities and third-party integrations. For agentic workflows calling multiple systems, each third party receiving ePHI may need its own assessment. Under the HIPAA Security Rule, a BAA must exist before any vendor touches ePHI.
Control What AI Agents Can Access and Invoke
Embedded AI features expand ePHI access within tools you already govern. AI agents are different: they act autonomously across system boundaries, often with elevated privileges. Access to AI agents must be governed the same way access to sensitive systems is governed: strict authentication, least-privilege principles, and audit trails.
Consider an API Gateway Layer to Govern Shadow AI
Shadow AI is hard to stop through policy alone. The more effective control is at the API and integration layer that sits between your systems and the outside world, monitoring and enforcing what data can leave, in what form, and to which destinations. The underlying data never leaves the governed environment, materially reducing the blast radius of shadow AI disclosure.
How StratoKey Helps HIPAA Organizations Govern and Secure ePHI Across a Growing AI Stack
StratoKey's Cloud Data Protection platform helps HIPAA-regulated organizations address the three governance challenges AI creates: protecting ePHI before it reaches AI systems, controlling how data moves across integrations and APIs, and governing which users and machines can access it.
Data Protection of ePHI Before Transmission to SaaS
AI features embedded in SaaS platforms only ever receive what you send them. StratoKey's Tokenization and Encryption features secure ePHI before it is transmitted to any SaaS platform or AI tool, replacing sensitive identifiers with tokenized or encrypted values.
Data Protection for APIs and Shadow AI
AI agents, agentic workflows and integrations move data across system boundaries, often without a human initiating each step. StratoKey's API Gateway enforces policy on that machine-to-machine communication and tokenizes or encrypts the payload before it is transmitted. This can also be used to limit exposure by shadow AI, the gateway can be configured to inspect and govern user based requests.
Identity and Access Controls
The Identity Gateway governs which users and systems can access ePHI and invoke AI agents. It enforces authentication and least-privilege controls at the access layer, ensuring that elevated privileges cannot be exploited.
Policies, Rules, and Audit Logs
Governance requires more than protection. It requires visibility. Across all three gateways, StratoKey enforces configurable rules and policies and maintains the audit logs that the proposed NPRM requires.
StratoKey Can Help Secure Your AI Use
StratoKey helps HIPAA-regulated organizations reduce ePHI exposure while still being able to benefit from AI capabilities. It secures ePHI before SaaS platforms (like NetSuite, ServiceNow, Salesforce, Jira and Confluence), AI tools, and integrations can access it in plaintext. If you are working through your AI compliance posture, get in touch.
AI and HIPAA Compliance: The Risks and How to Reduce Your Exposure
AI has moved into healthcare fast. Faster than most compliance programs have kept up. Covered entities and their business associates are applying AI..
Encryption of ePHI, a Required Safeguard for HIPAA Compliance
The U.S. Department of Health and Human Services (HHS) in January published a "Notice of Proposed Rule Making (NPRM) which includes significant..