“Thinking” artificial intelligence is now able to diagnose real-life conditions as well as or better than doctors, according to the results of a study published on Thursday Science.
The researchers used previously unknown clinical cases to test OpenAI’s o1 reasoning model against the company’s older model, the GPT-4, with doctors and medical residents in training.
In the testing range, the o1 model is often the most advanced in the diagnostic power of GPT-4 and the best clinicians, too. When tested on random emergency electronic health records from a Boston hospital, the o1 model was diagnostically accurate more than two-thirds of the time in the first test. The two medical specialists present had the correct diagnosis about half the time.
What AI can tell you about your blood test
Dr. Robert Wachter, professor and chairman of the Department of Medicine at the University of California, San Francisco, described the findings of the study as “significant” and suggested that it is now “undeniable” that modern AI will surpass older models and doctors when they are asked to identify the correct diagnosis and the next step. He was not participating in the research.
However, Wachter, author of “The Giant Leap: How AI Is Transforming Healthcare and What It Means for Our Future,” he added that more research is needed before AI can be fully implemented in clinical practice.
Mashable Light Speed
“The question is how well this replicates real life, and the answer is yes but not well,” Wachter wrote in an email.
As the study’s authors admit, the test was limited to text only and did not include the visual and auditory clues and clues that doctors often rely on to make a diagnosis. This can include the patient’s level of depression and the medical picture.
“GenAI can probably start to synthesize these inputs but for now, a clinical case study that is written, and often ‘clean’ by itself is not the same as walking into the ER and dealing with chaos,” Wachter said. “Just look Pitt.”
When is ‘The Pitt’ Season 3 coming out?
Based on their findings, the study authors highlighted the “urgent” need for additional studies and potential clinical trials to determine how AI systems can improve clinical practice and patient outcomes.
“The rapid pace of development in LLMs has a major impact on the science and practice of medicine,” wrote the authors, most of whom are based at Boston’s Beth Israel Deaconess Medical Center, where the study was conducted.
An accompanying article, also published in Science and written by two experts at the Flinders Health and Medical Research Institute in Adelaide, Australia, who were not involved in the study, agreed with its immediate results. They also argued against replacing doctors with AI, instead envisioning a collaborative style that provides guidance, contextual judgment, and accountability.
Without demonstrated robust performance, equity, and safety, many AI systems will remain inadequate for clinical use,” the experts wrote.
