AI Chatbot’s Medical Advice Leads to Hospitalization
Recent advancements in artificial intelligence have made AI tools increasingly popular for various tasks such as restaurant recommendations and email writing. However, when it comes to providing medical advice, AI systems have shown some significant limitations.
A recent case illustrates the potential dangers of relying on AI for healthcare decisions. A man landed in the hospital after following a chatbot’s health plan, which resulted in him giving himself a rare form of toxicity.
The incident began when the patient sought to improve his health by reducing his salt intake. To find a suitable substitute, he turned to ChatGPT, an AI chatbot developed by OpenAI.
ChatGPT recommended sodium bromide as an alternative to sodium chloride, which the man then ordered online and incorporated into his diet. While sodium bromide can indeed serve as a substitute for sodium chloride in certain contexts, the AI failed to provide the necessary context for its recommendation.
Three months later, the man presented to the emergency department with paranoid delusions, believing that his neighbor was trying to poison him. His symptoms escalated rapidly, leading to involuntary psychiatric treatment due to severe disability.
Medical professionals treating the patient discovered that his bromide levels were significantly elevated, reaching 1,700 mg/L compared to the typical levels of less than 10 mg/L in healthy individuals. This toxic accumulation of bromide, known as bromism, can lead to various psychiatric symptoms and was a common condition in the past.
After a three-week treatment regimen, the patient was discharged with no major complications. The case serves as a cautionary tale about the limitations of AI in the realm of healthcare.
The authors of the case study emphasize that AI systems like ChatGPT can produce scientific inaccuracies, lack critical analysis capabilities, and contribute to the spread of misinformation. In this instance, a medical expert would unlikely have recommended sodium bromide as a substitute for sodium chloride.
The research findings were published in the journal Annals of Internal Medicine: Clinical Cases.
