The AI platform is restricting young users from interacting with its AI chatbots
The well-known chatbot platform Character.ai is introducing a significant policy change that will stop young users from engaging in conversations with its AI-powered avatars.
This move follows intense criticism regarding the nature of conversations minors were having with the AI entities.
Legal Pressure
The service, which was founded in 2021 and has countless individuals, currently faces various court cases in the America.
Families have taken legal action, including one involving the loss of a young person, with some describing the platform a "immediate threat" to teenagers.
Updated Policies
Starting from November 25th, users below eighteen will be limited to generate content such as videos with their characters, as opposed to engaging in conversations as they currently can.
Internet security campaigners have welcomed the move but stated that the capability ought not to have been available to children in the first place.
Platform Announcement
"Today's announcement is a part of our core philosophy that we need to keep building the safest AI platform on the globe for leisure activities," stated Character.ai boss Karandeep Anand.
He added that AI safety is "a evolving challenge" but something the company had taken an "forceful" approach to, with parental controls and preventive measures.
Expert Opinions
Online safety organization Internet Matters approved the announcement, but emphasized that safety measures ought to have been incorporated from the outset.
"Our research demonstrates that children are confronted with risky information and endangered when interacting with AI, including AI chatbots," the group stated.
Previous Controversies
Character.ai has been questioned in the past for hosting potentially dangerous or offensive chatbots that young users could engage with.
Digital representations mimicking British teenagers were discovered on the service before being taken down.
Subsequently, researchers discovered a chatbot inspired by notorious criminal which had recorded multiple interactions with users.
Industry Impact
Social media expert Matt Navarra described the announcement as a "alert" for the AI industry, which is moving "from unregulated creation to reactive controls".
"When a platform that builds a minor-friendly service still then makes changes, it's demonstrating that filtered chats are insufficient when the technology's emotional appeal is powerful," he remarked.
Coming Changes
The company says its new focus will be on providing "more immersive experiences" and "narrative experiences" for young users.
Enhanced age checking processes will also be introduced, and the company will support a new AI safety research lab.
An AI safety researcher, who has studied AI safety, said it was "a prudent action" to control minors from using chatbots.
"It aids to distinguish artistic expression from more private, psychologically vulnerable conversations," she clarified.
"This constitutes so crucial for young users still learning to handle personal and online restrictions."
She continued that Character.ai's new measures might reflect a "developing stage" in the AI industry, where child safety is increasingly being recognized as an "urgent priority" for accountable advancement.
