Who signs the report when the AI is wrong
The conversation about AI in medicine is mostly about detection rates. The harder question, and the one procurement asks last, is who carries the risk when the model is wrong.
In radiology, a report carries a name. Someone reads the images, forms an opinion, and signs it, and from that moment they are accountable for it. That simple fact is easy to lose in the excitement about artificial intelligence, where the whole conversation seems to be about detection rates. I want to ask a less comfortable question. When an AI-assisted read is wrong, whose name is on it?
The question a medico-legal file asks
Part of my work is independent medico-legal reporting, which means I spend time in the files where a diagnostic decision went wrong and someone is now trying to work out who was responsible. Those files are rarely about what the scan showed. They are about what the responsible clinician knew, what a reasonable practitioner would have done, and who owned the decision. AI does not remove that question. It makes it harder to answer, because it inserts a new party into the chain who has an opinion but cannot be held to account.
The liability does not move to the vendor
The comfortable assumption is that buying a clever tool moves the risk to the company that built it. It does not. When something goes wrong, regulators and courts still look for the responsible person, and in almost every arrangement that person is the clinician who signed, or the leader who chose to deploy the tool. If a model misses a cancer and a radiologist signed the report, the radiologist carries it. If a triage tool quietly de-prioritised a case that was then never read, someone decided to trust that tool with that decision. The liability does not disappear when you sign the purchase order. It redistributes itself along a chain that procurement almost never maps.
Accountability has to be engineered, not assumed
So the responsible adoption of diagnostic AI is a governance question before it is a clinical one. The questions worth asking before a tool arrives are unglamorous and rarely asked first. Who signs the result. What the override path is, and whether a clinician can disagree with the machine without friction. What gets logged, so that a decision can be reconstructed later. How failure is detected once the tool is live, rather than only in the vendor’s validation study. Who is liable, whether that is insured, and what the vendor will actually indemnify. These belong on a procurement checklist and in a board paper, and they tend to be asked, if at all, after the contract is signed.
Automation bias hollows out the signature
There is a quieter failure that worries me more than any single wrong result. A confident machine erodes the human check that the whole accountability model depends on. Give a busy clinician a screen that says normal, and the pull is to agree and move on. The signature stays in place, but the judgement behind it thins out, until the human in the loop is a rubber stamp on the machine’s opinion. That is accountability on paper and not in practice, which is the most dangerous kind, because the system looks governed while nobody is really deciding.
The chain gets longer exactly where it matters most
This connects to the work I care about most. AI-assisted reading helps most in the places with the fewest radiologists, which is precisely where the access thesis points us. But those are also the settings where the accountability chain is longest and thinnest: a scan taken in a rural community, read remotely, with a model in the loop and the signing clinician a thousand kilometres away. The further the read sits from the patient, the more deliberately the accountability has to be built, because there is less standing around it to catch a failure by chance.
The question to answer before you buy
The field measures these tools by what they can detect, and that matters. But for anyone deploying one, the more useful test is whether you can say, in advance, who signs the report when it is wrong, what happens next, and who answers for it. If you cannot answer that before the tool arrives, you are not ready to buy it. Detection is the demo. Accountability is the thing you are actually taking on.