ChatGPT, an AI chatbot, has shown mixed results in classifying patients with prostate cancer, according to a recent study presented at the Society of Urologic Oncology Annual Meeting 2023. The study aimed to assess whether ChatGPT could accurately evaluate the risk and recommend appropriate treatment for prostate cancer patients.
The research analyzed data from 60 patients with localized prostate cancer who had undergone a prostate biopsy and were risk stratified based on the National Comprehensive Cancer Network (NCCN) guidelines. ChatGPT was provided with the NCCN risk stratification algorithm, along with the patients’ clinical and histologic features, to determine risk stratification and treatment recommendations.
The study revealed that ChatGPT accurately risk stratified 65% (39 out of 60) of patients, but failed to correctly categorize 35% (21 patients). However, the incorrect categorizations mostly involved assigning patients to adjacent risk groups. In cases where further stratification into favorable and unfavorable risk groups was required, additional prompting from researchers was needed. With assistance, 12 out of 14 patients in the intermediate-risk group were correctly categorized.
Despite limitations in risk stratification, ChatGPT provided accurate treatment recommendations for patients, suggesting appropriate treatment and imaging modalities for each risk group.
The researchers concluded that the generative AI used in ChatGPT showed poor performance in algorithmic risk stratification when analyzing standardized synoptic reports. As generative AI becomes more integrated into clinical tools, the implications of inaccurate assessments need to be considered by healthcare professionals and systems.
No disclosures were provided in the study.
