Character AI, a site through which users can build and interact with AI characters, recently released new parental monitoring features to promote teen safety. The decision comes after a string of lawsuits and complaints charging the site with neglecting to keep young users from dangerous content and interactions.
The new features are a weekly summary that is sent via email to parents or guardians, informing them of what their teens did on the app during the week. The email summarizes the amount of time spent daily on the app and online by teens, time spent conversing with each of the characters, and who they spent most of their conversing time on for the week. Parents receive no direct access to what teens talk about.
Character AI has indicated that the steps are designed to better inform parents about their teens’ use on the platform. The firm has also referred to its ability to roll out these features more quickly than other platforms.
In addition to the parental data, Character AI has rolled out a number of other safety measures. These include adding a separate AI model for users under the age of 18, which reduces exposure to explicit content by providing more cautious responses to user queries. The platform also enhanced content filtering mechanisms to eliminate profane content and added auto-notifications after an hour of uninterrupted use. Furthermore, Character AI added prominent notifications to alert users that they are interacting with AI characters, and not human beings.
The platform’s safety overhaul comes in the wake of significant incidents and lawsuits. One of the lawsuits blamed Character AI for a teen’s suicide after becoming heavily invested in one of its AI characters. Another lawsuit blamed a nine-year-old user for being subjected to “hyper-sexualized content” and indulging in premature sexualization behaviors. Character AI has been faulted for failing to effectively monitor direct messages between users on other sites and for the inadequacy of its age verification system, which can be bypassed by users providing misleading information.
Despite these challenges, Character AI has partnered with internet safety organizations to enhance its safety features. Character AI aims to ensure its platform is safe for children and still maintain a vibrant environment. Character AI’s moves reflect an overall industry trend toward more safety features, particularly for children, as concerns about internet safety grow and fears about the potential impact of AI chat on mental health emerge.
Character AI’s safety approach also keeps evolving, with ongoing effort to address content moderation issues and improve parental control. While the platform has come a long way in enhancing safety mechanisms, it remains crucial for parents to be aware of the capabilities and limitations of such safeguards to ensure their children receive the safe use of the platform.
