Study Uncovers Inaccuracies in Cancer Advice Provided by ChatGPT, Raising Concerns about Accuracy
Researchers from Brigham and Women’s Hospital, affiliated with Harvard Medical School, recently conducted a study that revealed significant errors in the cancer advice provided by ChatGPT, an advanced chatbot developed by OpenAI. This study, published in JAMA Oncology, highlighted instances where ChatGPT provided false information when users inquired about cancer and its treatments.
The findings of the study showed that ChatGPT often presented a mix of correct and incorrect data, making it challenging for users to distinguish accurate information from misinformation. The presence of both accurate and erroneous facts within the system’s responses posed a specific problem, as it undermined the vital role of clarity and reliability in addressing health-related concerns.
The study’s authors, including Dr. Danielle Bitterman, expressed their surprise at the level of incorrect information mixed with correct information in ChatGPT’s responses, making it difficult even for experts to identify errors. This revelation underscores the critical need for users to approach these AI technologies critically and seek advice from medical professionals rather than relying solely on automated systems.
While ChatGPT has become immensely popular for its wide range of functionalities and natural conversation abilities, including text writing and mathematical problem-solving, it must be noted that errors in health-related matters are of significant concern. Dr. Bitterman emphasized that, although large language models like ChatGPT can generate convincing-sounding responses, they are not designed to provide accurate medical assistance. The high error rate and response instability in critical health-related matters need to be addressed in a clinical setting to avoid potential harm.
It is worth mentioning that this study’s findings are contradictory to another study conducted in May, which praised ChatGPT for its empathetic responses in discussing diseases and treatments. However, the recent study revealed that 12.5% of ChatGPT’s responses were fabricated, and it was more likely to present incorrect information when asked about localized treatments for advanced diseases or immunotherapy.
The accuracy of automated systems in the field of medicine remains a subject of debate. A Harvard scientist claimed that GPT-4, the language model utilized by ChatGPT, could pass the medical licensing exam and possess better judgment than some human professionals. Nevertheless, the prevalence of errors identified in ChatGPT’s health-related responses raises valid concerns about patient safety and the dissemination of accurate medical information.
In light of these concerns, the US Federal Trade Commission (FTC) launched an investigation in July to determine whether OpenAI’s ChatGPT is disseminating misinformation. The investigation was prompted by a complaint filed by the Center for AI and Digital Policy, which emphasized the risks associated with relying on ChatGPT’s responses without independent evaluations.
OpenAI acknowledges the possibility of errors in ChatGPT in its terms and conditions and explicitly states that the model is not designed to provide medical information or replace professional healthcare advice. As the accuracy of generative artificial intelligence models remains in question, it is essential to approach these technologies and their outputs with caution.
Ultimately, while AI systems like ChatGPT have made remarkable advancements, they are unlikely to replace doctors in the near future. The emergence of errors in ChatGPT’s responses to health-related inquiries necessitates a critical evaluation of the technology and reinforces the importance of seeking guidance from medical experts to ensure accurate information and patient safety.
