Hospitals are deploying AI like crazy. We still don’t know if it helps patients.

I don’t need to tell you AI is everywhere. It’s in your phone, your car, your search engine, and increasingly, your hospital.

Doctors are using AI scribes to take notes during appointments. Algorithms are scanning patient records to flag people who might need extra support. AI is reading chest x-rays and interpreting lab results. A growing pile of studies says these tools are accurate. But here’s the question nobody seems to be asking with any rigor: does using them actually make patients healthier?

We don’t know. And that’s the problem.

Jenna Wiens, a computer scientist at the University of Michigan, and Anna Goldenberg from the University of Toronto just published a paper in Nature Medicine that calls this out directly. Wiens told me she spent the first decade of her career trying to pitch AI to clinicians. Nobody wanted it. Then, a few years ago, “a switch flipped.” Suddenly hospitals couldn’t get enough of the stuff. They started deploying AI tools at a pace that far outstrips our ability to evaluate them.

The issue isn’t that these tools are bad. It’s that we have almost no idea whether they’re actually helping.

Take ambient AI scribes. These tools listen to doctor-patient conversations, transcribe them, and generate summaries. Multiple products are on the market, and they’re being adopted fast. A staffer at a major New York medical center told me recently that doctors are “overjoyed” with them. The scribes let them focus entirely on the patient during appointments instead of typing notes. Early studies show reduced clinician burnout. Great, right?

But what about patient health outcomes? Wiens points out that the existing research mostly measures satisfaction—doctor satisfaction, patient satisfaction—not whether clinical decision-making actually improves. We don’t know if the scribes change what doctors recommend, or how they process information, or whether patients end up better off.

This applies to just about every AI tool in healthcare right now. Predictive models flag patients at risk of deterioration. Recommendation engines suggest treatments. Imaging AI speeds up diagnosis. But accuracy doesn’t automatically translate to better outcomes. A faster chest x-ray read doesn’t matter if the doctor doesn’t trust the AI’s analysis, or if it changes how they interact with the patient in ways we haven’t studied.

Wiens raised an interesting point about cognitive effects. Research on AI in education suggests that these tools can change how people process information. Could the same happen with doctors? Could an AI scribe subtly alter how a medical student thinks about patient data? We don’t know, because nobody’s looking.

A study published in January 2025 by Paige Nong at the University of Minnesota found that about 65% of US hospitals were using AI-assisted predictive tools. Of those, only two-thirds evaluated their accuracy. Even fewer checked for bias. That’s bad, and the number using AI has probably gone up since then.

Wiens isn’t anti-AI. She says she believes in its potential. But she wants hospitals and independent researchers—not just the companies selling these tools—to actually study what happens when they’re deployed. “I do believe in the potential of AI to really improve clinical care,” she told me. “I just want more information about how they are affecting people.”

This is higher than I expected, honestly. I assumed hospitals were being more careful. Turns out they’re running an experiment at scale, and nobody’s collecting the data.