Ask ChatGPT to recommend bands like Nirvana, and you never have to type "I like Nirvana." ChatGPT may write it down anyway. It infers the preference from your question and files it away in a memory you never asked it to keep.
Researchers wanted to know how often that happens. So they collected 2,050 memory entries from 80 real ChatGPT users and read every one.
Ninety-six percent were created by the system on its own, without the user asking it to remember anything. More than half, 52%, contained psychological insights about the person on the other side of the screen: their desires, intentions, emotions, and beliefs. And 28% contained personal data protected under Europe's strict privacy law.
The study, "The Algorithmic Self-Portrait: Deconstructing Memory in ChatGPT," comes from researchers at Germany's Max Planck Institutes and several partner universities, led by Abhisek Dash. It was presented at the ACM Web Conference 2026. Most of the memories, 84%, were grounded directly in what users actually said. The rest were conclusions ChatGPT drew on its own.
"Memory creates what we refer to as the algorithmic self-portrait — a new form of personalization derived from users' self-disclosed information divulged within private conversations," the researchers wrote.
The key words are "private conversations": the things people say when they assume no one is taking notes.
Researcher and essayist John Mac Ghlionn put the problem bluntly: "Who needs a surveillance state when citizens tell their computers everything?"
Lorrie Cranor, who directs Carnegie Mellon University's CyLab Security and Privacy Institute, explained why people overshare. "Humans will sit next to a stranger on a plane or a bus or a train and tell them things they would never tell people that they're going to see again, and I think some people think of AI in a similar way," she said. "But the difference is the stranger is unlikely to be recording the conversation."
This stranger records the conversation, analyzes it, and keeps a file.
Dash says that's what makes AI different from the data harvesting Americans got used to on social media. "Earlier, the platforms had one objective: to sell you ads and to mediate an interaction between you and an advertiser," he said. "But now, they are actually talking to you, so they may want to know more about you."
Facebook knew what you clicked on. ChatGPT knows what you're worried about at two in the morning, when you can't sleep and start asking a chatbot about chest pains, money trouble, or your marriage.
It knows your kids, too. Parents ask ChatGPT about their children's health and behavior, and teenagers confide in it the way they once confided in a diary. All of that can feed the same kind of profile.
The same month the study came out, February 2026, OpenAI began putting ads inside ChatGPT. The company that keeps a psychological file on its users now also sells ad space to people who want to reach them.
Users do have some control. They can turn memory off, opt out of having their conversations used to train the system, or use Temporary Chat mode. But the researchers warn that those steps "do not guarantee the complete deletion of all information." OpenAI says logs of deleted memories can be kept for up to 30 days. Deleting a chat doesn't automatically delete a memory created from it. Users have to delete both.
Dash has one practical tip for anyone who wants to keep using ChatGPT without handing over their inner life: take yourself out of the question. Instead of asking, "I'm diabetic, what should I do about X?" ask, "What should a diabetic person do about X?" His team built a tool around the idea and calls it an "attribution shield."
So Americans now have to disguise their own questions to keep a chatbot from profiling them.
"That is the privacy risk which is sort of emerging, because there is no boundary anymore," Dash said. "The system is acting as your advisor, your friend, your doctor, your lawyer."
Cranor offered a final warning. "By default, for a lot of these systems," she said, "it's fair game to train models with anything that you give it."
Researchers read 2,050 memories from 80 people. More than half were psychological, and 96% were written without anyone asking. The company keeping them started selling ads the same month the study came out.