Harvard University Announces OpenAI o1 Outperformed Doctors in Emergency Triage Diagnosis

The research team conducted an experiment involving 76 patients who visited the emergency department of a Boston hospital. The AI and two doctors referenced standard electronic health records, including vital signs and demographic information, under the same conditions. As a result, o1 provided an accurate or very close diagnosis in 67% of the cases, while the doctors remained at 50–55%.

The AI's superiority was particularly pronounced in triage scenarios where information is limited and rapid judgment is required. When detailed data were available, o1's accuracy rose to 82%. While this was not statistically significant compared to the 70–79% achieved by doctors, it remained at a high level.

It was also reported that o1 outperformed 46 doctors in creating long-term treatment plans. For five clinical cases, o1 achieved a score of 89%, whereas doctors using conventional resources such as search engines scored 34%.

However, lead researcher Arjun Manrai and others stated that AI is not intended to completely replace doctors. This study focused only on patient data that can be communicated via text and did not include the evaluation of visual signals, such as the degree of distress or physical appearance. Manrai explained that the AI serves a role similar to a clinician providing a "second opinion."

Another lead researcher, Adam Rodman, stated that over the next decade, AI will form a "triad care model" working alongside doctors and patients. The research also highlighted a case where the AI correctly identified a connection to lupus in a patient with pulmonary embolism that the doctors had overlooked.