Meta has introduced a new AI-driven safety feature that alerts parents when teenagers discuss suicide or self-harm with Meta AI. The feature expands the company's teen safety initiatives on Instagram and aims to help parents intervene before a mental health crisis escalates. The rollout begins in the United States, the United Kingdom, Australia, and Canada, with a global expansion planned later this year.
This system operates using the parental controls on Instagram. If Meta AI detects that a conversation indicates a teenager may be at risk of suicide or self-harm, a human reviewer will review the chat before a notification is sent to the parents.
In its statement, Meta indicated that it would prefer to be cautious about a conversation even if it is not clear than to miss a possible crisis. The alerts do not provide parents with access to their teen's conversations. They notify parents that their child may need support and offer expert-backed guidance on how to begin sensitive discussions about mental health.
Meta announced that Parents will receive alerts when their teens mention suicide or other forms of self-harm while using Meta AI on Instagram and Facebook. The social media giant confirmed that it will share expert resources to help parents approach these conversations with their teens.
Meta also confirmed it is developing a capability that could notify emergency responders if Meta AI detects an imminent risk of suicide. The feature will apply to both teens and adults. It's based on the emergency referral system that exists in the social network.
The recent update is an improvement of existing measures such as alerts sent to parents in case teens start searching for suicidal content on Instagram. At the same time, Meta AI provides links to crisis hotlines and prompts users to call someone they trust.
According to Meta, about 75 experts and mental health clinicians contributed to developing the chatbot's responses. As regulators continue to scrutinize AI-powered chatbots, Meta says it will keep expanding safeguards for young users while preserving their privacy.
Also Read: Meta Faces Lawsuit Over AI-Driven Layoffs, Says Humans Made Final Calls