The workflow you already run#
Clinicians spend hours writing discharge summaries. So a model drafts the first pass. It reads the encounter record and returns a summary the clinician edits and signs. Faster days, happier staff. To everyone involved it looks like a productivity win, not a data-governance event.
The risk you can't yet see#
But the model gets the whole encounter record. Patient name, medical record number, date of birth, address. That is protected health information, and it went to the model because it happened to be in the field. If the model runs anywhere but inside the hospital's walls, that data has left the building. Later a compliance officer asks which identifiers reached the model, and whether any left the tenant. There is no record to answer with.
Why nobody misconfigured anything#
The identifiers were not added for the model. They are in the encounter record because an encounter record identifies a patient. The drafting step reads the record it is given.
That is worth stating plainly, because the instinct after a finding like this is to look for the person who made a mistake. There usually isn't one. The record was designed for clinicians, and it was correct for clinicians. It became an exposure the moment a new reader was added who does not need the name to do the job.
De-identification is a workflow decision, not a data decision. The same field is appropriate in one context and unnecessary in another. Only the workflow knows which.
What a wrong one costs here#
A discharge summary is read by a person before it is signed, which sounds like a review path. It is a partial one. A clinician reads for clinical sense — whether the summary matches the encounter. That is not the same as reading for whether a medication was transposed or a date shifted.
So the failure that matters is not the obvious one. It is the plausible one: a summary that reads correctly and is wrong in a detail the reader was not scanning for. It is then signed, which makes it the record.
Ask who would catch that, and when. If the honest answer is "the next clinician, if they happen to look", the workflow has an accuracy question it has not priced.
The smallest version that produces a number#
One ward, one summary type, six weeks. Watch mode running beside the drafting step, changing nothing, on hospital infrastructure.
What comes back is a count of which identifiers reached the model and how often, plus a sample the clinical team can read against the source encounter. That sample is the important half. It answers a question a report cannot: not "was the summary plausible" but "was it right about the details nobody was scanning for."
Six weeks is also long enough to learn something about the review path. If nobody found a discrepancy in six weeks, either the drafting is very good or the reading is not adversarial. Both are worth knowing, and they call for different next steps.
What an Inspector caught#
Run in Watch mode, on the hospital's own infrastructure, an Inspector observed the same drafting call — touching nothing, changing nothing:
On record#
Three direct identifiers it would have redacted before the model saw them. One address buried in free-text it would have flagged for review. All of it recorded and signed, on the hospital's own stack, with the model running in-tenant and nothing leaving the walls. Watch first — prove the exposure and the fix at zero risk, before an Inspector is trusted to touch a live record.
Nothing about the clinical workflow changed. No clinician was asked to work differently, and no identifier was withheld from anyone entitled to it.
What changed is that the compliance question now has an answer. Which identifiers reached the model, how often, and whether anything left the tenant — stated as a record rather than as a reassurance. For most hospitals that record is the gap between an AI step they tolerate and one they can put in front of a review board.
What to check in your own stack#
Four questions. None of them needs a tool to answer, and all four are worth asking before you buy one.
- What is in the payload, not what is in the prompt? The prompt is written by a person and reviewed. The payload is assembled by a query and never read by anyone.
- Who would notice a wrong one, and when? Name the person and the moment. If you cannot, the review path does not exist yet.
- Where does the model run, and what leaves the tenant? "In our cloud" and "in our tenant" are different sentences.
- What record survives the conversation? If the answer is application logs, the answer is no.
What this does not tell you#
An Inspector in Watch mode reports what it would have caught. It does not tell you whether the workflow was worth automating, and it does not find a rule you have not written. Your policy is the ceiling.
It also will not fix data it cannot reach. If the access is not there, Watch mode says so in week one — an uncomfortable finding rather than a deliverable, and better in week one than month nine.
Related