Microsoft researchers have developed an artificial intelligence tool that outperformed general physicians in diagnosing complex medical cases, according to a study published in the journal Science on Thursday. The system was more than twice as accurate as doctors in assessing difficult cases and identifying the likely diagnosis.

The research examined an AI model that follows a step-by-step diagnostic process. Instead of analyzing all available patient information at once, the system determines which questions to ask, which examinations to conduct and which tests to order.

The approach uses an ‘orchestrator’to coordinate a simulated panel of clinical experts. The system updates its working hypotheses as new information becomes available and decides whether further tests are needed.

Researchers evaluated the model on more than 300 unusual and challenging medical cases. It performed better than general physicians who used textbooks and online resources to reach their diagnoses.

The tool also outperformed doctors when tested on scenarios drawn from a major database of emergency room admissions. Researchers said the findings offered a more realistic assessment of medical AI than conventional tests based on multiple-choice questions.

"AI has become very good at getting a 100% ​pass rate on those,” said Dr. Dominic King, the corporate vice president of the health unit at Microsoft AI.

“But of course, that's not real-world. No ​patient ever came to see me with a nice summary,” he added.

The researchers also found that diagnostic work-ups designed by the AI orchestrator cost less than those conducted by human physicians. The study assessed both diagnostic accuracy and how carefully the system selected investigations.

“We're not just measuring the accuracy of the systems. We're also measuring how judiciously and carefully it goes through a ⁠diagnostic ​work-up,” said Harsha Nori, head of AI at Microsoft AI Health.

The next phase will involve a collaboration with Beth Israel Deaconess Medical Center in Boston. Researchers plan to assess whether the technology can help triage patients waiting in emergency department waiting rooms.

The upcoming tests will examine the system in a clinical setting, extending the research beyond controlled diagnostic cases.

Also Read: Meta Introduces AI Safety Tool to Flag Teen Mental Health Risks