Neil asked, "Do you (or other HIFA members) have any examples of incorrect responses [by AI chatbots] to requests for healthcare information?"
So I asked ChatGPT to "Give me a list of papers in which chatbots make health errors". Below is the result. ChatGPT listed four papers from genuine, reliable sources (links given). Three of the studies concluded with significant doubts about the reliability of using chatbots for specific health tasks, while one of them was positive. Below is the complete investigation. I read all four papers and quote their conculsions. My conclusion is that it is hard to be confident in the value of chatbot responses to health problems.
ChatGPT said: Several studies have investigated instances where AI chatbots, such as ChatGPT, have provided inaccurate or unreliable health-related information. Here are some notable examples:
1. Bias and Inaccuracy in AI Chatbot Ophthalmologist Recommendations: This study found that AI chatbots frequently recommended physicians who were not ophthalmologists or were located outside the specified city. Additionally, there was a significant bias against recommending female ophthalmologists. PMC
CZ: Conclusions were “This study revealed substantial bias and inaccuracy in the AI chatbots’ recommendations.”
2. Accuracy of Vitreoretinal Disease Information From an Artificial Intelligence Chatbot: Researchers discovered that only 15.4% of responses from an AI chatbot regarding vitreoretinal diseases were completely accurate. Some responses included potentially harmful medical advice, such as incorrect treatment options for conditions like epiretinal membrane and central serous chorioretinopathy. JAMA Network+1hcplive.com+1
CZ: Results were “Only 8 (15.4%) of the 52 questions submitted initially were graded as completely accurate.” Also “Ophthalmologists need to be aware of the limitations and potential for dissemination of misinformation associated with these AI platforms.”
3. Accuracy of Prospective Assessments of 4 Large Language Model Chatbot Responses to Patient Questions About Emergency Care: This study evaluated AI chatbot responses to emergency care questions and found that the advice was often incomplete and inaccurate, posing potential risks to patients who might rely on such information without proper medical consultation. JMIR
CZ: Conclusions: “AI chatbots, though ubiquitous, have significant deficiencies in EM patient advice, despite relatively consistent performance. Information for when to seek urgent or emergent care is frequently incomplete and inaccurate, and patients may be unaware of misinformation. Sources are not generally provided. Patients who use AI to guide health care decisions assume potential risks. AI chatbots for health should be subject to further research, refinement, and regulation. We strongly recommend proper medical consultation to prevent potential adverse outcomes.”
4. Accuracy and Reliability of Chatbot Responses to Physician Questions: An analysis revealed that AI-generated answers to physician inquiries were inconsistent over time. While some responses improved in accuracy upon reevaluation, others remained incorrect, highlighting concerns about the reliability of AI chatbots in medical contexts. hcplive.com+2PMC+2JAMA Network+2
CZ: Conclusion “Chatbot shows promise as a tool for providing accurate medical information in clinical settings.”
These studies underscore the importance of exercising caution when relying on AI chatbots for medical information, as inaccuracies and biases can lead to misinformation and potential harm.
Chris Zielinski
Centre for Global Health, University of Winchester, UK and
President, World Association of Medical Editors (WAME)
Blogs; http://ziggytheblue.wordpress.com and http://ziggytheblue.tumblr.com
Publications: http://www.researchgate.net and https://winchester.academia.edu/ChrisZielinski/
HIFA Profile: Chris Zielinski held senior positions at the World Health Organization for 15 years, in Africa, WHOs Geneva Headquarters, and India, and earlier in other UN-system organizations working in writing, media, publishing, knowledge management, and intellectual property. He also spent three years as Chief Executive of the Authors Licensing and Collecting Society (looking after the intellectual property revenues of all UK authors and journalists). Chris was the founder of the ExtraMED project (Third World biomedical journals on CD-ROM), and managed the Gates Foundation-supported Health Information Centres project. At WHO he was appointed to the Ethical Review Committee, and was an originator of the African Health Observatory during his years in Brazzaville. With interests in the information, and computer ethics and bioethics, Chris has edited numerous books and journals and worked as a translator. Now working independently, Chris has recently finished writing a travel book called Afreekinout. Email: chris AT chriszielinski.com