AI and Your Personal Data: What Companies Know
Quality reviewed September 2026
Facts verified September 20, 2026
- Subject
- Career & Life Skills
- Grades
- Grades 9-12
- Length
- 9-page student worksheet
- Time
- About one 50-minute class period
- Great for
- Sub plans and independent work
- Includes
- Reading passage, comprehension questions + full answer key
- Format
- Print-and-go PDF
What students do
- Five terms, including data broker and the mosaic effect, get matched to their definitions
- A specific number from the Mozilla investigation and the age COPPA protects both get written into the reading's blanks
- Answer why AI combined with personal data works differently than an ordinary database
- Write out the specific privacy risk that applied to Carlos before he understood what the app's policy allowed
What it covers
- Data brokers and the largely unregulated market for personal profiles
- The mosaic effect and what AI can infer from data that seems harmless
- A case study on a teen's mental health app conversations and who else can see them
Start reading this worksheet
The Data That Feeds the Machine
Every interaction you have with a digital device generates data. When you search for something online, the search terms are recorded. When you watch a video, the duration, the pauses, the rewinds are logged. When you use a navigation app, your location, speed, and route are tracked. When you shop online, your purchases, browsing history, and even your cursor movements may be collected. When you use a social media platform, your posts, likes, comments, follows, and the amount of time you spend looking at different types of content are all data points. When you interact with an AI assistant or chatbot, the content of your conversations may be stored and used. This data is extraordinarily valuable: not just for improving services, as companies often explain it, but as a commercial asset. The behavioral data you generate helps companies build detailed profiles of who you are, what you want, how you make decisions, and how you are likely to behave in the future. Those profiles are used to target advertising, but they are also increasingly used to power AI systems that make decisions about you in contexts far beyond advertising: your creditworthiness, your insurance risk, your suitability for a job, your likelihood of defaulting on a loan, whether you should receive extra scrutiny from law enforcement.
What AI Systems Actually Collect
AI assistants and chatbots represent a particularly intimate form of data collection. When someone types a question into an AI system, they often share things they might not tell a friend, a doctor, or a family member (health symptoms, relationship problems, financial worries, questions about sexuality, struggles with addiction). The conversational format creates a sense of confidentiality that may not actually exist. OpenAI's terms of service for ChatGPT, as of 2024, state that the company uses conversation data to train future versions of the model. Users can opt out of some of this data use, but the option is buried in settings and not the default. AI-powered apps in health, mental health, fertility, and personal finance collect even more sensitive data. Period-tracking apps collect menstrual cycle data. Mental health apps collect therapy session notes. Personal finance apps have access to bank accounts. Research has found that many of these apps share data with third parties, including advertisers and data brokers, even when their privacy policies claim otherwise. A 2021 investigation by researchers at Mozilla Foundation found that many popular mental health apps had privacy practices that fell far short of their stated policies.
The Data Broker Ecosystem
Much of the data that AI systems use to build profiles of individuals does not come directly from users' interactions with those AI systems. It comes from data brokers, companies whose entire business model is collecting, aggregating, and selling personal information. Data brokers buy data from apps, retail loyalty programs, public records, social media platforms, and other sources. They combine information from multiple sources to build comprehensive profiles: your name, address, age, estimated income, political affiliation, religion, health conditions, purchase history, and location patterns. All of it is compiled into a single dossier and sold to whoever is willing to pay. Data brokers are largely unregulated in the United States. The Federal Trade Commission (FTC) has called for stronger regulation, but as of 2026, no comprehensive federal data broker law exists. In 2023, a data broker sold location data from a dating app that was used to identify and out the location of a Catholic priest, who subsequently resigned. In 2021, Motherboard reported that data brokers were selling location data of people who visited Planned Parenthood clinics. These cases illustrate that data originally collected for one purpose (matching people with dates, tracking foot traffic at retail locations) can be repurposed in ways that harm individuals, and that the subjects of data collection have little ability to prevent it.
Like what you read? The full reading, the case study, and every question set are in the free download below, along with the answer key.
Why this matters
Most of my students hand over data every day without thinking about where it lands. This reading gives them Carlos's situation, a kid who told an app about his anxiety and his immigration worries, and asks them to name the actual risk in plain terms. It also puts a number on the problem: no federal privacy law covers most of what they do online in 2026, so the choices they make about which apps to trust are choices with real, lasting consequences.
From one teacher to another
Every scroll, search, and chat leaves a trace, and this reading walks students through where all that data actually goes. It opens with the ordinary stuff, search terms, video pauses, location pings, then moves into the AI chatbot layer, where OpenAI's ChatGPT terms allow conversations to train future models unless a user finds the buried opt out. From there the reading gets into data brokers, the largely unregulated companies that buy and resell personal profiles, including the 2023 case where a data broker's location data outed a Catholic priest through a dating app. Students meet the mosaic effect, the idea that harmless pieces of data become revealing once an AI assembles them, and they work through a case study about Carlos, a 16 year old who trusted a mental health app's chat bot with things he had not told anyone else.
What you get
Download
Where should I send the next one?
Everything here is free and it stays that way. Leave your email and I'll send you the new lessons as they go up. You can unsubscribe from any one of them.
Related resources
AI in Criminal Justice: Predictive Policing Risks
AI in Healthcare: Helping or Hurting Patients?
Facial Recognition and Wrongful Arrests
Should Schools Use AI to Watch Students?
Stolen Voices and Stolen Art: Copyright in the AI Age
The Right to Know You're Talking to a Bot
Every lesson on this site is free to download and use. If one of them saved you a prep period, you can chip in a few bucks toward my three kids' daycare fund. That's genuinely where tips go.