AI Consult Tool Proves Safe in Kenyan Primary Care but Does Not Yet Improve Short-Term Patient Outcomes

Sixteen primary care clinics in Kenya, more than 9,600 patients, and one question that most AI studies in medicine have avoided: when a clinician gets help from a generative AI tool, do the patients actually do better? A randomized controlled trial published in Nature Medicine offers a first answer. The tool was safe and it improved the quality of clinical decision-making. It did not significantly change short-term patient outcomes.

The work was delivered by researchers at the University of Birmingham, supported by the NIHR Biomedical Research Centre: Birmingham. It is described as one of the first randomized trials anywhere to test generative AI against patient-level outcomes, rather than measuring clinician performance or running simulated cases.

The design assigned clinicians, not patients, at random. Some used their electronic medical record system as usual. Others used the same system with an integrated tool called AI Consult, a large language model built directly into the record. While a consultation was under way, it worked in the background. It analysed what the clinician entered, produced diagnostic and treatment suggestions tied to Kenyan national clinical guidelines, and flagged potential concerns with a simple color code of green, yellow or red. Clinicians were not required to follow its advice and remained responsible for every diagnosis, prescription and referral. Patients could not see the interface, so the usual interaction in the consulting room was preserved.

The headline outcome was treatment failure within 14 days. It occurred in 2.2% of patients seen with AI support and 2.0% of those receiving standard care, a difference that was not statistically significant. Hospitalisation and death rates were similar in both groups, and the researchers found no evidence of harm. Patient satisfaction was the same too, which the team reads as a sign that the tool did not change how patients experienced their care.

Where the tool did show an effect was in the work itself. An independent panel of experienced clinicians, blinded to whether AI had been used, judged that clinical documentation and treatment planning were significantly better in the AI-supported group. Overall antibiotic prescribing rates were similar between groups, but antibiotic-related costs were lower with AI support, because of more cost-conscious prescribing choices.

Professor Bilal Mateen, senior author and Chief AI Officer at PATH, called the results reassuring but also sobering. The technology appears safe and clearly improves aspects of decision-making, he said, but turning those gains into measurable patient benefit is much harder, particularly in everyday primary care. Part of the reason is statistical. Serious outcomes such as hospitalisation or death are rare in primary care, so detecting modest effects could require studies of more than 100,000 patients.

Professor Alastair Denniston, a co-author, added that much of primary care involves common and often self-limiting conditions, where patients need little intervention. In that setting, even meaningful improvements in clinical reasoning may produce only small changes in outcomes that are very difficult to measure. What the study does show, he said, is that AI can be integrated safely into real clinical workflows without undermining patient trust or clinician autonomy, which he called a critical foundation for any future impact.

Although the trial took place in Kenya, the researchers stress that the findings matter globally, including for high-income systems. Professor Richard Riley, a senior author, cautioned that whether the results carry over to wealthier settings, where baseline standards of care are already high, still needs to be evaluated. The study was funded by the Gates Foundation, sponsored by PATH, and conducted with the London School of Hygiene and Tropical Medicine and the KEMRI-Wellcome Trust Research Programme in Kenya. The next questions are how large future trials must be, and whether the same pattern holds where care is already strong.