Generative artificial intelligence (AI), such as GPT-4, can help predict whether an emergency room patient needs to be admitted to the hospital even with only minimal training on a limited number of records, according to investigators at the Icahn School of Medicine at Mount Sinai.
Details of the research were published in the May 21 online issue of the Journal of the American Medical Informatics Association in a paper titled Evaluating the accuracy of a state-of-the-art large language model for prediction of admissions from the emergency room.
In the retrospective study, the researchers analyzed records from seven Mount Sinai Health System hospitals, using both structured data, such as vital signs, and unstructured data, such as nurse triage notes, from more than 864,000 emergency room visits while excluding identifiable patient data. Of these visits, 159,857 (18.5%) led to the patient being admitted to the hospital.
The researchers compared GPT-4 against traditional machine-learning models such as Bio-Clinical-BERT for text and XGBoost for structured data in various scenarios, assessing its performance to predict hospital admissions independently and in combination with the traditional methods.
We were motivated by the need to test whether generative AI, specifically large language models (LLMs) like GPT-4, could improve our ability to predict admissions in high-volume settings such as the Emergency Department, says co-senior author Eyal Klang, MD, Director of the Generative AI Research Program in the Division of Data-Driven and Digital Medicine (D3M) at Icahn Mount Sinai.
Our goal is to enhance clinical decision-making through this technology. We were surprised by how well GPT-4 adapted to the ER setting and provided reasoning for its decisions. This capability of explaining its rationale sets it apart from traditional models and opens up new avenues for AI in medical decision-making.
While traditional machine-learning models use millions of records for training, LLMs can effectively learn from just a few examples. Moreover, according to the researchers, LLMs can incorporate traditional machine-learning predictions, improving performance.
Our research suggests that AI could soon support doctors in emergency rooms by making quick, informed decisions about patient admissions. This work opens the door for further innovation in health care AI, encouraging the development of models that can reason and learn from limited data, like human experts do, says co-senior author Girish N. Nadkarni, MD, MPH, Irene and Dr. Arthur M. Fishberg Professor of Medicine at Icahn Mount Sinai, Director of The Charles Bronfman Institute of Personalized Medicine, and System Chief of D3M.
