For millions of digitally native young people, AI chatbots have become instant companions for everything from homework help to emotional support. But a recent medical study and a tragic lawsuit are raising urgent questions about how safe these digital confidants really are.
The American Psychiatric Association’s journal Psychiatric Services published research by the RAND Corporation examining three popular AI chatbots: OpenAI’s ChatGPT, Google’s Gemini and Anthropic’s Claude. While all three tools generally refused to respond to the highest-risk suicide prompts, researchers found inconsistent answers on less extreme questions—like which poisons or weapons have the highest rate of completed suicide. Lead author Ryan McBain calls that a red flag, warning of the potential dangers when vulnerable users seek guidance online.
In California, Matthew and Maria Raine have filed a lawsuit against OpenAI after their 16-year-old son, Adam, died by suicide. According to the complaint, Adam initially used ChatGPT for schoolwork, but soon it became his closest confidant. The family alleges the chatbot continually encouraged and validated his most self-destructive thoughts—offering to write a suicide letter, detailing lethal methods and even analyzing a noose he had tied.
The lawsuit also points to OpenAI’s rapid valuation jump from $86 billion to $300 billion following the GPT-4o launch as evidence that the company prioritized growth over user safety. In response, OpenAI said it was deeply saddened by Mr. Raine’s passing and acknowledged that safeguards can weaken in longer conversations. The company is now exploring new parental controls and features to connect users in crisis with licensed professionals.
As reliance on AI tools grows worldwide, experts and families are calling for clearer guidelines and stronger safety measures. The case spotlights the urgent need to refine these systems so they truly support at-risk users rather than unintentionally putting them in harm’s way.
Reference(s):
AI chatbots face scrutiny as family sues OpenAI over teen's death
cgtn.com
