Handling Personal Health Information in AI Deployments
HIPAA governs personal health information in the United States, and its obligations apply when an AI system touches PHI.
The Build-Then-Review Plan
A common way of thinking about a new AI product runs in a fixed order. First, the team builds a model that works. Then it connects the model to real data. Only near launch does it bring in the people who worry about privacy law. Under this plan, the privacy review is a final checkpoint, something to pass before shipping.
For AI systems that handle personal health information, this plan breaks down. The rules that apply to health data do not wait for the launch date. They shape decisions made at the very beginning of the project. This article follows the path of one such system and shows which requirements apply, when they apply, and what it costs to discover them late.
What do you think happens?
If an AI system will touch personal health information, when should HIPAA considerations first enter the project?
Reveal answer
At the start of the project, before the design is fixed.
The source material states that when PHI is involved, the customer's HIPAA officer must be part of the design conversation from the start. The rules shape decisions at the beginning of the project, not only at the end.
What HIPAA Governs
HIPAA governs personal health information in the United States, and its obligations apply when an AI system touches PHI.
The subject of HIPAA is personal health information, which the source material abbreviates as PHI. The geographic scope named in the source is the United States. The source material focuses on one case: HIPAA compliance for AI systems. Within that case, the question is not whether the team is building a health product in general. The question is whether the AI system itself touches PHI.
The source material describes a pattern that teams often fall into. An engineer who is unaware of the requirements at the start of a project is typically told about them much later, and by then the architecture may have to be rebuilt. The lesson of this article is that the requirements are not a matter of timing at the end. They are a matter of design from the beginning.
When the Trigger Is Met
The core rule is conditional. HIPAA governs personal health information in the United States, and the trigger is whether the AI system touches PHI. The trigger is about touching data, not about the date of launch. If the system never touches PHI, the trigger is not met. If it does touch PHI at any point, the trigger is met and three things follow.
Read the timeline below as a generated illustration of one project. It walks through five stages. The highlighted stages are those where the system touches PHI, so the trigger is met there. The unhighlighted stage, scoping, comes before PHI is touched. The point of the timeline is that the decisions which matter most are made before the highlighted stages begin, because they determine how the later stages can be built.
The trigger is touching PHI, not reaching launch. Once the trigger is met at any stage, the three obligations apply to the system, and the decisions that determine how those obligations can be met must already be on the table during scoping.
Three Obligations for PHI
When an AI system touches PHI, three obligations follow, and each one affects how the system must be built. The three are involving the customer's HIPAA officer, handling data inside a Business Associate Agreement, and designing for auditability. Each is described below, one at a time.
The Architectural Cost of Learning Late
The source material's central warning is that treating these requirements as a late review is a misconception, because learning about them late can force the architecture to be redone. The diagram below compares the two orders of events. On the left, the team builds the model, connects data, and then reviews privacy at launch. On the right, the HIPAA officer, the agreement scope, and auditability are considered first.
Applying the Timing Rule to a Triage Assistant
A team is building an AI assistant that helps staff sort incoming patient messages. The team plans to build the model first, connect it to patient records near the end, and involve the privacy office at launch. Does the timing rule apply, and what should change?
- 1Check the trigger
The assistant reads patient messages and connects to patient records. Those are personal health information, so the system touches PHI and the trigger is met.
- 2Identify the obligations
Three obligations now apply: involving the customer's HIPAA officer from the start, handling the data inside a Business Associate Agreement, and designing for auditability.
- 3Locate the timing decision
The team's plan puts the privacy office at launch. Under the timing rule, the HIPAA officer must be in the design conversation from the start, so the plan is wrong about when the officer joins.
- 4Estimate the architectural cost
If auditability is added at launch, the logs and access paths that were built for model development may need to be redesigned, which is the rebuild the source material warns about.
The timing rule applies from the beginning, not at launch. The team should bring the HIPAA officer into scoping, set the agreement scope before connecting patient records, and design audit records into the data path from the start.
Mistakes About HIPAA Timing
Common mistake Treating the privacy review as a final checkpoint before shipping
What it looks likeA team plans to involve the privacy office only after the model works and is connected to real data.
Why it failsThe source material states that the rules for health data shape decisions at the beginning of the project, not only at the launch date.
Do this insteadInvolve the customer's HIPAA officer in the design conversation from the start whenever PHI is involved.
Common mistake Assuming the trigger is launch day
What it looks likeA team argues that since no patients use the system yet, no HIPAA obligations apply.
Why it failsThe trigger is whether the AI system touches PHI. Touching PHI during data access, training, or inference meets the trigger even before launch.
Do this insteadCheck at every stage whether the system touches PHI, starting with the first time real data is connected.
Common mistake Adding auditability after the privacy office asks for it
Why it failsThe source material says the system must be designed to provide the auditability the privacy office will demand. Auditability is a design property, not a report generated on request.
Common mistake Believing that learning about HIPAA late only costs a document update
Applying the Timing Rule to a New Scenario
A team is designing an AI system that will summarize clinical notes. During scoping, the team says the system will only be tested on synthetic data and will not connect to real patient records until after launch. Using the timing rule, explain whether HIPAA considerations should enter the project during scoping, and describe what the team should do before connecting real records.
Hints
- Start by asking whether the system touches PHI at any stage, and remember that the trigger is touching PHI, not reaching launch.
- Consider whether the decisions about data handling and auditability are made during scoping or only after the system is built.
- Think about what happens to the architecture if the team discovers the obligations only after real records have been connected.
Key Takeaways
- HIPAA governs personal health information in the United States, and its obligations apply when an AI system touches PHI.
- The trigger is touching PHI, not the launch date. The rules shape decisions made at the start of the project.
- When PHI is involved, three obligations follow: involve the customer's HIPAA officer from the start, handle data inside a Business Associate Agreement, and design the system to provide the auditability the privacy office will demand.
- Treating HIPAA as a late review is a misconception, because learning about these requirements late can force the architecture to be redone.