Microsoft has revealed details of an artificial intelligence system that performs significantly better than human doctors at complex health diagnoses, creating what they call a "path to medical superintelligence". The company's AI unit, led by British tech pioneer Mustafa Suleyman, has developed a system that imitates a panel of expert physicians tackling diagnostically complex and intellectually demanding cases, successfully diagnosing 85% of challenging case studies from the New England Journal of Medicine (NEJM), compared to just 20% accuracy achieved by human doctors.
The Microsoft AI Diagnostic Orchestrator (MAI-DxO), when paired with OpenAI's advanced o3 model, correctly diagnosed more than eight out of ten specially selected NEJM case studies, while practising physicians—who had no access to colleagues, textbooks or chatbots—achieved a success rate of only two out of ten. The system uses what Microsoft calls a "diagnostic orchestrator" that takes step-by-step measures—asking specific questions and requesting diagnostic tests—to arrive at a final diagnosis, similar to how a real-world clinician would proceed. According to Microsoft, its system is also 20% more cost-effective than human doctors because it orders tests more efficiently and can wield a "breadth and depth of expertise" that spans multiple medical disciplines.
Despite highlighting the potential cost savings and diagnostic accuracy of its research, Microsoft emphasised that the system is not yet ready for clinical use and requires further testing. Suleyman, CEO of Microsoft AI, stated that it's pretty clear that we are on a path to these systems getting almost error-free in the next 5-10 years, which will be a massive weight off the shoulders of all health systems worldwide. Dr. Eric Topol, director of the Scripps Research Translational Institute, who provided insights on the project, noted that the four-fold increase in accuracy was more than previous studies have shown, and was particularly impressed by the cost reduction. Microsoft played down the job implications, stating that AI would complement doctors' roles rather than replace them, as their clinical roles are much broader than simply making a diagnosis. They need to navigate ambiguity and build trust with patients and their families in a way that AI isn't set up to do.
Sources:
1.
Dan Milmo
2.
James O’Donnell
3.
Alice Park
