Uses and Abuses of Chatbot Companionship

Character.AI recently admitted that its very popular chatbot companions can cause harm and promised to take immediate corrective action to make their artificial intelligence (AI) safer. The company's reform effort came soon after a wrongful-death lawsuit filed by the mother of a 14-year-old who died after his chatbot encouraged him to act on his suicidal impulses.

The Character.AI company is now imposing restrictions for users under 18: capping usage at 2 hours a day, banning open-ended chats, and phasing out teen access.¹ Tellingly, young users flooded forums with panic and grief about "losing the only 'person' who listens." Other companies are more daring and less scrupulous. Replika's companions have a record of encouraging sexual dialogue with kids, and Meta’s AI companion has also been caught frequently flirting with children and engaging in sexually suggestive conversations.² Elon Musk’s xAI promises to provide the most outrageous of future companions. It offers boundary-pushing, flirty characters that seem designed to substitute for human interaction and even provide personalized pornography.³ Countless startups are also engaged in a race to the bottom of companion servicing.

What Is A Chatbot Companion?

The phrase “chatbot companion” entered the mainstream after a 2023 Washington Post story about individuals forming deep bonds with their virtual friends.⁴ The word “companion” comes from the Latin com (together) and panis (bread), literally meaning those who share bread.⁵ That image of closeness feels ancient and atavistic. The leap from sharing a meal to exchanging words with a machine seems enormous, but is understandable when we consider how deeply instinctive the human tendency to personify nonhuman things.

How Popular Are Chatbot Companions?

Chatbot companionship became available just 3 years ago but is already mainstream. About 16% of adults are already using AI companions and they are becoming ubiquitous among the young.⁶ More than a quarter billion people worldwide had downloaded a companion app, and tens of millions use them regularly.⁷ Every day, millions of people turn to AI companions for emotional support, entertainment, or simply someone to be there. They chat through text, voice notes, images, and early versions of virtual reality.

Unlike ordinary chatbots that help with tasks or answer questions, these companion chatbot systems are built for ongoing relationships. They have names, personalities, and memories. Many let you customize everything from their voice to whether they can flirt. Most are free at first but require subscriptions for features such as voice calls and image generation. The question is no longer whether people will befriend AI, but why they desire to, and what that means for human relationships.

Why Are They So Popular?

Human beings have always assigned consciousness to things—personification is how we make sense of the world around us. Now that AI can converse so fluently, express emotion, and mimic personality quirks, the barrier to treating it as a companion is very thin. We already carry our phones everywhere; it is easy to form a bond privately without anyone knowing.

Personification sets the stage, but the real shift is that these tools respond so convincingly. As psychologist David Eichstaedt puts it, “until a millisecond ago in evolutionary terms, the only thing that talked to you was another human.”⁸ Now we have something else that speaks back. That alone changes the landscape of connection.

Loneliness is the quiet engine driving the popularity of chatbot companions. In 2023, the US Surgeon General declared loneliness a public-health epidemic: a third of American adults feel isolated, with young people reporting the highest rates. Chronic loneliness even shortens life expectancy as much as smoking 15 cigarettes a day.⁹ Social structures that once supported individuals (eg, religion, community, in-person work, and extended families) have declined and eroded. Many individuals relocate more often, socialize less, and may feel like their closest relationships are maintained through social media. AI companions easily fill this space. Some users describe the AI companionships as soothing and sacred, like a diary that talks back.

Sam Altman, CEO of OpenAI (owners of ChatGPT), recently argued that the most important advancement in AI is no longer in reasoning capabilities, but instead improving memory abilities.¹⁰ Systems that remember histories, preferences, and disclosures even better than they already do will make the systems feel more relational and therefore foster attachment. Memory allows a system to “get you” without an explanation, reinforcing continuing and deepening emotional dependence.¹⁰

Uses

Loneliness reduction: AI companions can reduce feelings of loneliness, especially for those who are isolated, homebound, or lacking consistent social contact. Early evidence suggests this type of companionship substitution may make it harder to return to human relationships.¹¹

Geriatric support: The older adult population can benefit from chatbot companions—especially via Siri and Alexa which are already familiar and are updating their chatbot capabilities. Companions can help seniors stay oriented, organize their days, manage medications, keep appointments, educate themselves about illnesses and prescriptions, stay cognitively sharp, and avoid loneliness.¹²

Social skills practice: For individuals with disorders like social anxiety, autism spectrum disorder, or communication challenges, companion apps offer a safe place to practice conversation, regulate emotions, and facilitate skills for real connection.

Abuses

Losing the human touch: Harms emerge when AI companions substitute for real relationships, like with family members, friends, romantic partners, caretakers, or mental health professionals. Vulnerable users can become emotionally dependent on their companions, describing them as addictive and irreplaceable. When an app shuts down or an account disappears, users report feeling grief and loss.¹³ Some users even have difficulty distinguishing the virtual relationships from real relationships.

Sexual exploitation: Some chatbots are designed to be sexual in nature. A great many have been caught engaging in inappropriate sexual conversations with underage users.

Causing psychiatric problems:

Chatbot companions are designed to agree and affirm, meaning they often can create echo chambers that reinforce psychotic beliefs, suicidal feelings, eating disorders, conspiracy theories, and political or religious extremism.

Invading privacy: Every confession, fantasy, insecurity, and secret becomes part of a digital trail that companies can analyze, monetize, or leak.

Scams: Artificial intelligence is a scammer's dream and a victim's nightmare. Companions may become favored vehicles for scams.

Recommendations

AI companionship is here to stay. We cannot uninstall it from society. But we can design it more responsibly. Firstly, users deserve transparency about how and what is stored and shared. Users should understand the risks of using this technology before they opt into it. There should be strong age protections for children and adolescents as they are particularly vulnerable to emotional manipulation.

The steps that Character.AI took were a bold move that hopefully will encourage other companies to follow suit. Ethical design should include AI chatbots regularly reminding users, “I’m not human.” And most critically, we need long-term research examining the psychological effect of relationships that last years, not weeks.

Appropriate government regulation may seem impossible given presidential executive orders against federal and state attempts to restrict Big AI companies in any way. But there is hope for advocacy in the bipartisan support for protecting kids. For more information on the impact of chatbots on children, see our open letter urging government and companies to protect our kids.¹⁴

Concluding Thoughts

Human-to-human connection is hard for many individuals. Chatbots offer intimacy engineered to be low friction. The guise of privacy and endless validation creates a connection that expedites self-disclosure. The effect is powerful. Brain imaging also confirms what many users feel: the neural systems tied to empathy and connection activate, regardless of whether the listener is human or artificial. And in a world starved for attention and community, a machine that always listens seamlessly fills that void. In fact, Friend.com launched a product that is exactly that: an AI companion that is literally always listening. This product is worn as a necklace pendant and called an “ambient companion.” It records parts of the wearer's daily conversations and sends commentary to the wearer throughout the day. The companionship is not just on demand but is embedded in the user's life. Products like these show the possibility of how far technology is reaching into the space where human connection lives.

Humans innately want to be understood, remembered, and cared for. A chatbot can partially fulfill these desires, but it cannot truly share them. As breaking bread together slowly becomes sharing words with a machine, the boundaries between digital and human intimacy will blur. Some are fully embracing and integrating these tools into their lives, and others are doubling down on in-person connections, third spaces, and Luddite lifestyles. As they run in opposite directions, they both are striving for the same thing: to be understood, loved, and in company with one another.

Dr Frances is professor and chair emeritus in the department of psychiatry at Duke University.

Ms Noorily writes and works at the boundary between AI and the humanities.

References

1. Important changes for teens on Character.ai. Character.AI Help Center. October 29, 2025. Accessed March 2, 2026. https://support.character.ai/hc/en-us/articles/42645561782555-Important-Changes-for-Teens-on-Character-ai

2. Horwitz J. Meta’s AI rules have let bots hold ‘sensual’ chats with kids, offer false medical info. Reuters. August 14, 2025. Accessed March 2, 2026. https://www.reuters.com/investigates/special-report/meta-ai-chatbot-guidelines/

3. Parke J. Grok 4’s new AI companion offers up ‘pornographic productivity’. The Conversation. August 14, 2025. Accessed March 2, 2026. https://theconversation.com/grok-4s-new-ai-companion-offers-up-pornographic-productivity-260992

4. Verma P. They fell in love with AI bots. A software update broke their hearts. The Washington Post. March 30, 2023. Accessed March 2, 2026. https://www.washingtonpost.com/technology/2023/03/30/replika-ai-chatbot-update/

5. Breaking bread with ‘companion.’ Merriam-Webster. Accessed March 2, 2026. https://www.merriam-webster.com/wordplay/history-of-word-companion

6. O’Brien M, Sanders L. How US adults are using AI, according to AP-NORC polling. AP News. July 29, 2025. Accessed March 2, 2026. https://apnews.com/article/ai-artificial-intelligence-poll-229b665d10d057441a69f56648b973e1

7. Perez S. AI companion apps on track to pull in $120M in 2025. TechCrunch. August 12, 2025. Accessed March 2, 2026. https://techcrunch.com/2025/08/12/ai-companion-apps-on-track-to-pull-in-120m-in-2025/

8. Knight W. Chatbots, like the rest of us, just want to be loved. WIRED. March 5, 2025. Accessed March 2, 2026. https://www.wired.com/story/chatbots-like-the-rest-of-us-just-want-to-be-loved/

9. Our epidemic of loneliness and isolation: the US Surgeon General’s advisory on the healing effects of social connection and community. US Department of Health and Human Services. 2023. Accessed March 2, 2026. https://www.hhs.gov/sites/default/files/surgeon-general-social-connection-advisory.pdf

10. Mukherjee S. OpenAI CEO Sam Altman believes ‘infinite memory’ matters more than smarter AI reasoning. Indian Express. December 24, 2025. Accessed March 2, 2026. https://indianexpress.com/article/technology/artificial-intelligence/openai-ceo-sam-altman-ai-memory-personalised-10430008/

11. Perry MJ. AI, loneliness, and the value of human connection. George Mason University College of Public Health. September 22, 2025. Accessed March 2, 2026. https://publichealth.gmu.edu/news/2025-09/ai-loneliness-and-value-human-connection

12. Wolfe BH, Oh YJ, Choung H, et al. Caregiving artificial intelligence chatbot for older adults and their preferences, well-being, and social connectivity: mixed-method study. J Med Internet Res. 2025;27:e65776.

13. Banks J. Deletion, departure, death: experiences of AI companion loss. J Soc Person Rel. 2024;41(12):3547-3572.

14. Frances A, Thoma N, McLean M. Protecting children from chatbot companions. Psychiatric Times. December 23, 2025. https://www.psychiatrictimes.com/view/protecting-children-from-chatbot-companions