In the emergency departments of one large health system during 2025, a machine-learning tool woven into the electronic record began quietly raising a hand whenever a patient's chart suggested opioid use disorder. The effort rests on a long and painful problem: opioid overdoses kill tens of thousands of Americans each year, and emergency rooms often stand as the only door a person at risk ever walks through. Medicines that could steady such patients are rarely offered there, and the researchers suspect the first obstacle is simply finding the people who need them. Their findings show a tool tha
Machine-Learning Tool Flags Likely Opioid Use Disorder in Emergency Department Patients in Real Time
Very good when it fires, but it likely misses a substantial share
Mark
Why does a flag in the chart matter so much for opioid treatment?
Mimi
Because the ED is often the only place these patients meet healthcare, and only about 6 to 8% of eligible patients get buprenorphine there. If clinicians don't recognize the patient in a busy shift, nothing else follows.
Mark
How well did the flags hold up when doctors checked?
Mimi
Of 217 reviewed patients, 92 flags were true and 11 were false. Positive predictive value came out at 0.893.
Luke
Careful with that number. The sample was stratified, roughly half model-positive by design. The authors themselves say it isn't population-level screening accuracy.
Mark
So what does the population picture look like?
Mimi
Weighted back up, specificity is about 0.996, but sensitivity drops to 0.40.
Luke
And that interval runs from 0.24 to 0.64. It rests on just five false negatives. So the tool may miss many patients, and the reporting can't say how many with any precision.
Mark
Is the alert volume manageable?
Mimi
About 77 a day across three EDs, which is what the threshold was chosen to limit.
Luke
That's a projection from 2025 flag rates, though, not a measured clinician experience. Nothing here says how staff responded.
Mark
And the calibration issue?
Mimi
Predicted probabilities were underestimated, and nobody recalibrated before deployment.
Luke
Which matters if anyone reads the score as a literal probability. Whether any of this raises buprenorphine starts is unknown. The earlier tool didn't.
Der Puls
- A tool flagged roughly 77 emergency visits a day across three hospitals, and each alert asks a busy clinician to notice a patient who might otherwise slip past unseen.
- Only about 6 to 8% of eligible ED patients receive buprenorphine, while overdose deaths keep climbing, so the gap between need and treatment feels urgent and widening.
- Physicians re-checked 217 charts against the model, and the tool proved sharp when it fired, with a positive predictive value near 0.89, yet it caught perhaps only 40% of true cases in the wider population.
- An earlier decision support tool from the same team failed to raise treatment rates, so the ADAPT trial now tests whether real-time flags can finally move patients toward care.
- The tool has not been recalibrated and its probabilities run low, so the team must tune its sensitivity before its alerts can reliably reach the people who need them.
In the emergency departments of one large health system during 2025, a machine-learning tool woven into the electronic record began quietly raising a hand whenever a patient's chart suggested opioid use disorder. The effort rests on a long and painful problem: opioid overdoses kill tens of thousands of Americans each year, and emergency rooms often stand as the only door a person at risk ever walks through. Medicines that could steady such patients are rarely offered there, and the researchers suspect the first obstacle is simply finding the people who need them. Their findings show a tool that is reliable when it speaks but silent about a meaningful share of those it should reach, a reminder that recognizing a need and meeting it are different acts.
Across three emergency departments in one large health system, 866,569 visits passed through in 2025. A machine-learning tool built into the electronic health record flagged 28,284 of them, about 3.26%, as possibly involving opioid use disorder. That came to roughly 77 alerts a day. The research team has now reported how those flags held up when physicians checked them against charts.
The underlying problem is grim and familiar. More than 109,000 American drug overdose deaths occurred in 2022, and nearly seventy percent involved opioids. Emergency departments are often the first or only healthcare contact for people at high risk. Buprenorphine can be started right there, and it saves lives, yet only about six to eight percent of eligible patients receive it in the ED. Prescribers' inexperience, worries about workflow and diversion, and lingering stigma all play a part.
The team had tried a remedy before. Their earlier EMBED trial built a decision support tool for starting buprenorphine, and it did not raise initiation rates. The ongoing ADAPT trial aims to do better, on the argument that finding the right patients is a necessary first step. The new tool was trained on adult ED visits from 2014 to 2025, covering over three million patients, and it performed well on retrospective data, though its predicted probabilities were underestimated and it was deployed without recalibration.
The real test came from a stratified sample of 217 charts from 2025, which independent reviewers adjudicated with strong agreement. There, the model's positive predictive value was about 0.89, meaning its alerts were usually right. Once the results were weighted back to the full population, however, sensitivity fell to roughly 0.40. The tool is trustworthy when it speaks, but it likely misses many patients who truly have the disorder.
The open question is whether better identification changes care. The earlier tool did not, and the ADAPT trial will show whether real-time flags can finally move patients toward buprenorphine.
Across the three emergency departments of one large health system in 2025, 866,569 eligible visits passed through the doors. A machine-learning tool built into the electronic health record flagged 28,284 of them, or 3.26%, as likely to involve a patient with opioid use disorder. That works out to roughly 77 alerts a day across the three sites. A research team has now reported on how well those flags held up when physicians checked them against patient charts.
The problem the tool addresses is a familiar one. In 2022 more than 109,000 drug overdose deaths occurred in the United States, and nearly 70% involved opioids. Emergency departments are often the first or only point of healthcare contact for people with opioid use disorder who are at high risk of overdose. Buprenorphine, a medication for the disorder, reduces illicit opioid use and opioid-related deaths and can be started in the ED. Yet only about 6 to 8% of eligible patients receive it there. The authors point to prescribers' lack of experience with the drug, worries about workflow and diversion, and persisting stigma.
The same group had already tried a fix. Their earlier EMBED trial built a clinical decision support tool for starting buprenorphine in the ED, and it did not raise patient-level initiation rates. The ongoing ADAPT trial, a Multiphase Optimization Strategy trial, aims to improve on that. The authors argue that accurately identifying potentially eligible patients is a necessary step. Earlier computable phenotypes could pick out patients with opioid-related disorders from chart data, but only retrospectively. Other models predicted well in retrospective settings but were rarely woven into clinical workflow.
To build the new tool, the team used adult ED visits from 2014 to 2025, covering 3,177,664 unique patients. After balancing the outcome, the training set held 17,054 patients and the test set 4,246. A random forest classifier was trained on a high-specificity computable label. At the default 0.50 threshold it scored a positive predictive value of 0.945, a negative predictive value of 0.958, sensitivity of 0.549 and specificity of 0.997. The confusion matrix showed 202 true positives, 10 false positives, 3,805 true negatives and 172 false negatives. The most influential individual predictors were diagnoses of substance use disorders other than opioid use disorder. By category, past medical history mattered most, followed by medications and laboratory results. The calibration curve showed predicted probabilities were underestimated, especially at the higher end, and no recalibration was done before deployment.
The prospective check came next. From the 2025 encounters, the team drew a stratified sample of model-positive and model-negative cases for independent physician review. They selected 200 patients, added 20 to allow for exclusions, and dropped three: one under 18 and two who were pregnant. That left 217 patients, of whom 103 were adjudicated as having opioid use disorder and 114 as not. Inter-reviewer agreement was strong, with a Cohen's kappa of 0.86. Resident reviewers disagreed on 17 cases. Attending physicians initially sided with the resident reviews on 13 of them, and the remaining 4 were settled by discussion.
Against that gold standard, the model produced 92 true positives, 110 true negatives, 11 false positives and 5 false negatives. That gives a positive predictive value of 0.893 (95% CI 0.823 to 0.942) and a negative predictive value of 0.957 (95% CI 0.907 to 0.983). The authors caution that these figures depend on the stratified sampling and should not be read as population-level screening accuracy.
When they weighted the results back to the full population, the picture was more modest. Estimated prevalence was 7.1% (95% CI 4.5 to 11.9), specificity 0.996, and sensitivity 0.40, with a wide interval of 0.24 to 0.64. That interval comes from the five false negatives among 114 model-negative cases, a proportion then applied to 96.5% of the source population. In plain terms, the tool is very good at being right when it fires, but it likely misses a substantial share of patients who have the disorder.
The open question is whether better identification changes treatment. The earlier EMBED tool did not increase buprenorphine initiation, and the ADAPT trial will test whether real-time flags help close that gap.
Bemerkenswerte Zitate
The validation estimates depend on the stratified sampling design and should not be interpreted as population-level screening accuracy.— Study authors
Möchten Sie die ganze Geschichte?
Das Original lesen bei PLOS (Public Library of Science) ↗
Health & Medicine