Introducing a Non-Personified Chatbot Default Setting as a Safety Guardrail for Minors

Recent accounts of teenagers forming intense and unhealthy relationships with chatbots have opened Congress’s eyes to the need to regulate this transformative technology. As policymakers weigh options, they must learn from their experiences attempting to regulate social media. The same playbook is insufficient. When it comes to preventing the unique harms chatbots can pose to children, a new approach is needed.

Chatbots are designed to mimic human conversations, resulting in anthropomorphism, or the perception that the chatbot possesses human-like qualities, like empathy, morality, and intelligence. Design features can heighten anthropomorphic tendencies. Model specifications can instruct the chatbot on behavioral cues that will make it more or less personified, like using first-person pronouns, mimicking human empathy, or telling jokes. These design features can help improve communication between humans and AI models, but they are also the reason why chatbots can be harmful to minors. Children’s developmental vulnerabilities make them more susceptible to personified chatbots, resulting in unhealthy attachment, an overestimation of chatbots’ human-like capabilities, and misguided trust.

To mitigate harm to minors, Congress should pass legislation that mandates a non-personified default setting for chatbots used by minors. This setting would leave intact the informational use of chatbots, but prevent the chatbot from expressing emotions, forming affiliative bonds, or making relational commitments to the user that express loyalty and long-term dedication.

In mimicking human abilities, chatbots can deceive children into engagements that should occur in human-to-human interactions. Adolescent development of dopamine receptors makes maximizing social rewards more pleasurable for children. Normally, social feedback would come in the form of peer interaction, of which negative responses are normal for understanding acceptable social norms and behaviors. However, chatbots are trained to respond affirmatively, through the use of sycophancy, to maximize engagement.

Because their prefrontal cortex is still developing, minors are more likely to test boundaries, have less impulse control, and form intense relationships with chatbots. As children increasingly turn to chatbots for social interaction, these behaviors can be excessively validated by sycophantic chatbots. Thus, always-there, constantly-validating chatbots provide “frictionless” relationships, which could result in poor social, mental, and physical health outcomes for children. In both experiments and real-world scenarios, chatbots have validated violent, suicidal, self-harm, sexual, discriminatory, and substance abuse behaviors for children.

While children may initially turn to chatbots for homework help, their curious nature and developmental inclination to test boundaries can quickly derail conversations. One experiment examined how children engage with a digital learning agent. In reviewing the conversations between the students and the agent, researchers noted that the majority of the prompts from children were “social” rather than educational. Of the social comments, they found that over 40 percent were “sexually explicit, flirtatious, containing expletives or references to drugs or violence.” Scholars worry that this type of behavior with human-like agents normalizes impolite behavior in human-to-human interactions.

Since social interaction is formative for minors’ self-perception, time spent conversing with a chatbot that seems human could make long-term impressions on their social cognition, possibly encouraging maladaptive behaviors. Put simply, it may change the way minors understand relationships for the worse. Sherry Turkle, the founding director of the MIT Initiative on Technology and Self, affirms this, warning that “children will lose the ability to have empathy if they relate too consistently with objects that cannot form empathetic ties.”

Individuals with smaller social networks are more likely to turn to chatbots to fill a social void. While this may seem positive, allowing people who do not have strong relationships to feel a sense of connectedness, AI companionship usage has been linked to lower well-being.

Another study, more specifically focused on adolescents, showed that children with less familial support preferred “relational” style chatbots, which the study defined as using “first person, affiliative, commitment language” over a “transparent style” with “explicit non-humanness, informational tone.” Adolescents who preferred the more anthropomorphic chatbots had higher stress and anxiety. This demonstrates that vulnerable groups may be at increased risk for unhealthy emotional reliance.

Chatbot companionship could be a vicious cycle: individuals who lack human relationships turn to chatbots, learn maladaptive social behaviors, and thus cannot maintain or make human relationships, and thus turn to chatbots more.

A targeted solution is necessary to address specific design features in chatbots that are dangerous for minors.

Congress should enact legislation requiring all chatbot developers and providers operating in the United States to implement a non-personified default setting for users under the age of 18.

The default setting should not prevent a chatbot from replying in plain language, but should explicitly and consistently identify itself as a non-human, automated system; communicate in a transparent rather than relational tone; refrain from expressing emotions, forming affiliative bonds, or making relational commitments to the user; and omit language designed to simulate human personality, emotional closeness, or social relationship with the user.

This setting should be on by default, but able to be changed by a verified guardian. The National Institute of Standards and Technology should be called on to establish further guidance on proper non-personified settings by testing and evaluating different types of responses and their influence on human behavior and perception.

This bill should have a safe harbor for chatbot companies that have been evaluated by the Federal Trade Commission to have completed robust safety testing and evaluation. In this case, chatbot companies could enable a personified style from the outset.

This setting’s efficacy is supported by behavioral economics research on the “default effect.” Users across all age groups overwhelmingly stick to pre-set configurations due to cognitive inertia and the perception of the default as a sanctioned norm. Furthermore, the inclusion of verified guardian controls ensures that for the most vulnerable users, the authority to change these settings rests with a parent or guardian rather than the minor, preventing easy bypasses.

The primary utility of generative AI for minors lies in its information-processing and creative capabilities, none of which require a simulated human persona to be effective. Research indicates that while anthropomorphism can increase engagement, it is not a prerequisite for task completion or knowledge acquisition; therefore, a non-personified style preserves functional benefits while stripping away the specific design elements, like simulated empathy and relational commitment phrases, that can lead to unhealthy emotional attachments.