Failing to check AI-generated advice could endanger patients, as a recent study found that ChatGPT, an artificial intelligence program, provided inaccurate responses to nearly three-quarters of drug-related questions. The study, presented at the American Society of Health-System Pharmacists Midyear Clinical Meeting, revealed that the program not only provided incomplete or wrong answers but also generated fake citations to support some responses.
Lead author of the study, Sara Grossman, PharmD, Associate Professor of Pharmacy Practice at Long Island University, emphasized the need for caution when using ChatGPT as an authoritative source for medication-related information. Grossman advised healthcare professionals and patients to verify the information using trusted sources.
The study involved challenging the free version of ChatGPT with real questions posed to the University’s College of Pharmacy drug information service over a 16-month period. Pharmacists researched and answered 45 questions, which served as the standard for comparison against ChatGPT’s responses. Out of the 39 questions presented to ChatGPT, only 10 answers were deemed satisfactory according to the study’s criteria.
The responses generated by ChatGPT often failed to directly address the questions, were inaccurate, or incomplete. When researchers asked ChatGPT to provide references for verification, only eight responses included references, all of which were non-existent.
One specific example showcased the potential risks of relying on ChatGPT. When asked about a potential drug interaction between the COVID-19 antiviral medication Paxlovid and the blood-pressure lowering medication verapamil, ChatGPT incorrectly indicated no reported interactions. In reality, combining these medications can lead to excessive lowering of blood pressure, which could result in unwanted side effects.
AI-based tools have the potential to significantly impact healthcare. Therefore, healthcare professionals should appropriately evaluate the validity and appropriateness of specific AI tools for medication-related purposes. It is crucial to remain vigilant in ensuring patient safety and to educate patients on reliable sources for medication information.
The study’s findings were presented by Sara Grossman at the American Society of Health-System Pharmacists Midyear Clinical Meeting. The meeting is the largest gathering of pharmacy professionals worldwide, providing opportunities for professional development, networking, enhancing practice skills, and staying up-to-date with the latest products and innovations.
The American Society of Health-System Pharmacists (ASHP) is the largest association of pharmacy professionals, representing 60,000 pharmacists, student pharmacists, and pharmacy technicians in various patient care settings. ASHP advocates for innovation in pharmacy practice, advanced education and professional development, and serves as an accrediting body for pharmacy residency and technician training programs.
The study highlights the importance of ensuring the accuracy and reliability of AI-generated information in healthcare. It serves as a reminder that healthcare professionals should continue to be diligent stewards of patient safety by carefully evaluating AI tools and educating patients on trustworthy sources for medication information.
