The case raises further concerns about AI safety for children as tech companies face more lawsuits over their technology.
The parents of a 13-year-old girl have filed a lawsuit against Google, alleging that their daughter used its AI technology on a school-issued computer to plan a possible suicide and her funeral.
ADVERTISEMENT
ADVERTISEMENT
Filed on Monday 2 November in a court in the US state of California, the lawsuit is believed to be the first to accuse Google of negligence over AI-powered search features accessed through a school-issued Chromebook, one of the company's low-cost laptops widely used in schools.
The girl, identified as Emma, was unable to access Google's Gemini app or OpenAI's ChatGPT, as both had been blocked by her school. However, she could still access Google's search engine, which includes AI-powered features.
Despite having "no history" of mental illness, Emma began a "quiet spiral into a severe psychological crisis" within months of using the search engine, according to the complaint.
On 17 March, the school alerted her parents after its monitoring software flagged a document titled "Funeral". Her mother later found her writing a farewell letter, prompting her father to intervene.
"Schools should turn search off and keep it off until Google removes AI mode from search or ensures that every single student is always safe when they're interacting with AI," Andrew Liddell, one of the family's attorneys, told AFP on Thursday 5 November.
Growing legal scrutiny of Big Tech
Google says its AI Mode includes safeguards against self-harm and attempts to bypass them.
"We are reviewing the claims in the lawsuit and we take this very seriously," Google spokesperson Jose Castaneda told AFP.
OpenAI faces several wrongful death lawsuits, including one filed by the parents of 16-year-old Adam Raine, who died by suicide after allegedly receiving harmful advice from ChatGPT.
In January, Google and Character.AI also agreed to settle several lawsuits brought by families of teenagers out of court.
Meanwhile, in a separate legal battle, Google, TikTok and Meta filed lawsuits against California on Wednesday to block a law banning addictive social media features for users under 16.
Signed by California Governor Gavin Newsom in September, the law is set to take effect on 1 January 2027.
