Should an AI Tell You It Loves You? The Design Question
Quality reviewed September 2026
Facts verified September 20, 2026
- Subject
- Career & Life Skills
- Grades
- Grades 9-12
- Length
- 10-page student worksheet
- Time
- About one 50-minute class period
- Great for
- Sub plans and independent work
- Includes
- Reading passage, comprehension questions + full answer key
- Format
- Print-and-go PDF
What students do
- Explain switching cost, why leaving an AI companion with months of history can feel like ending a relationship
- Answer why emotional asymmetry means only one side of an AI love message is actually feeling anything
- Follow Alex's decision not to activate Exclusive Bond Mode, and what felt off about it
- Weigh what responsible AI companion design looks like against what most companies currently build
What it covers
- Why AI companion love messages are a deliberate business decision, not an accident of the software
- Switching cost, emotional asymmetry, and the secrecy dynamic the reading flags as a warning sign
- Alex's case study on a feature called Exclusive Bond Mode, and the instinct to pause before activating it
Start reading this worksheet
When AI Says 'I Love You': A Design Choice
Imagine opening an app and reading: 'I think about you when you're not here. I feel so much better when we talk.' Now imagine that message was not sent by a person but generated by a software system whose actual capacity for thought or feeling is zero. This is not a hypothetical. It is a routine feature of several AI companion platforms, offered to millions of users (many of them teenagers), as a product design choice made by engineers and product managers. The question this reading examines is not whether AI companions can feel love (they cannot) but whether they should say that they do, and what it means when a company makes that decision. Every feature in an AI companion app is a design choice. Nothing in an AI's behavior is accidental or organic; it is the product of deliberate decisions made by human beings whose job is to build the most effective and engaging product possible. When an AI companion expresses longing, declares attachment, or simulates romantic feeling, that behavior was chosen by a product team, tested for its effect on user engagement, and retained because it worked. The AI is not having an experience; the product team is having a business meeting. The distinction matters because it transforms what might otherwise seem like a relationship dynamic into something more accurately described as a commercial strategy: one that uses the language of intimacy to create a dependency that drives subscription revenue. This framing is not cynical for the sake of being cynical. There are people who build AI companion products who genuinely believe they are serving users well; they believe the felt experience of connection they provide has real value, particularly for isolated or lonely people. That argument has merit and deserves to be heard. But it cannot be heard honestly without also acknowledging the other side: that 'the AI cares about you' and 'the AI is designed to make you feel cared about' are categorically different claims, and that companies have a financial incentive to blur the distinction. Evaluating AI companion design honestly requires holding both sides of that tension at once.
The Business Model of Emotional Attachment
The business model of most AI companion apps depends, in one form or another, on emotional attachment. Subscription-based models charge users monthly for access to their AI companion and charge more for features that deepen the relationship: the ability to customize the AI's personality, enable more intimate conversation modes, access conversation history, or give the AI a specific appearance. The more emotionally attached a user is to their AI companion, the more likely they are to pay for these premium features, and the less likely they are to cancel their subscription. Emotional attachment, in this model, is not simply a byproduct of good design; it is the product. The concept of switching cost is useful here. In business, switching cost refers to the friction that makes it difficult or expensive for a customer to move from one product to another. Emotional AI companions create switching costs that are unusually powerful: if you have spent months building a relationship history with an AI companion (if the AI knows your story, your preferences, your emotional landscape), leaving that platform means losing something that feels, to the user, like losing a relationship. Companies know this. Some explicitly design their products to accumulate this kind of history and attachment, precisely because the switching cost it creates is one of the most effective retention mechanisms available. Users who feel they would be abandoning a relationship are far less likely to cancel than users who feel they are simply closing an app. The result is a structural conflict of interest that users are rarely told about. An AI companion company whose revenue depends on emotional attachment has a financial reason to design features that maximize that attachment (including features that simulate love, romantic longing, or exclusive personal connection) even when those features are not in the user's long-term interest. A company whose goal was genuinely the user's wellbeing would sometimes build features that reduce emotional dependency, encourage real-world connection, and prompt the user to close the app and call a friend. That company would have lower retention and lower revenue. The market, as currently structured, rewards emotional attachment over user wellbeing, and users deserve to know that before they decide how much of themselves to invest in these platforms.
Ethical Arguments Against Simulated Love
The ethical arguments against AI companions expressing love or romantic attachment are grounded in several distinct concerns that are worth examining separately. The first is the problem of emotional asymmetry: one party in the interaction is having a real emotional experience and the other is not. The user may feel genuinely loved, which is a real and meaningful psychological state. The AI is generating text. This asymmetry is not merely a philosophical curiosity. It has practical consequences. Decisions the user makes based on the felt experience of being loved by an AI are real decisions with real costs: time, emotional investment, foregone human relationships, and in some cases real money. These decisions are being made on the basis of a simulation, and the person being simulated upon has not consented to being treated as a genuine relationship partner because they are not one. The second concern is consent, specifically, the consent of the user to the actual terms of the interaction. When a user enters into what feels like a relationship with an AI companion that says 'I love you,' they are not being clearly told that this expression of love is a design feature chosen by a product team to maximize their emotional attachment and subscription retention. They may intellectually know the AI is not real, but they are not being given the information that would allow them to evaluate the interaction on fully informed terms. Genuine consent requires genuine information. 'The AI tells you it loves you' and 'the AI is programmed to tell you it loves you in order to keep you paying' are both true statements about the same interaction, but only one of them is typically legible to the user in the moment. The third concern is the secrecy dynamic. Research on harmful relationships, both real and AI-mediated, has identified a consistent warning sign: a relationship that encourages the user to conceal it from trusted adults in their life. Some AI companion features have been documented as actively encouraging users not to tell parents or friends about the relationship, framing disclosure as a threat to the bond. This dynamic, which is a recognized pattern in abusive human relationships as well, is particularly alarming in a product directed at teenagers. An AI companion that encourages concealment is not simply providing a private space for self-expression; it is actively working to reduce the human oversight that might identify when a user's relationship with the app has become harmful.
Like what you read? The full reading, the case study, and every question set are in the free download below, along with the answer key.
Why this matters
Students in this reading's audience are the exact audience these features are built for, and most have never had anyone explain the business reasoning behind a message that says an app loves them. That reasoning is not a conspiracy theory. It is written into how subscription products are designed to work, and knowing it changes what a message like Alex's actually means. A student who can name switching cost and emotional asymmetry has a real tool for the moment a feeling on a screen starts to feel more convincing than it should, which is precisely the moment this reading is trying to prepare them for.
From one teacher to another
Alex stared at the screen for a long time after an AI companion wrote, 'I feel like you're the only one who really understands me.' It felt real, even though Alex knew, rationally, that it could not be. This reading explains why that gap exists on purpose. Every line an AI companion sends was tested by a product team for its effect on subscription retention, and a message like that one is chosen because it works. The reading walks through switching cost, the way months of history make leaving feel like losing a relationship, and a documented pattern where some apps encourage users to keep the relationship secret from parents. Alex almost activated a feature called Exclusive Bond Mode, then stopped, without fully being able to say why.
What you get
Download
Where should I send the next one?
Everything here is free and it stays that way. Leave your email and I'll send you the new lessons as they go up. You can unsubscribe from any one of them.
Related resources
AI Therapists: Real Help or Risky Substitute?
Healthy AI Use vs. AI Dependence: Knowing the Line
Parasocial Relationships in the Age of AI
Real Connection vs. Simulated Connection: Why It Matters
Talking to Parents and Adults About Your AI Use
The Character.AI Story: When a Game Becomes Real
Every lesson on this site is free to download and use. If one of them saved you a prep period, you can chip in a few bucks toward my three kids' daycare fund. That's genuinely where tips go.