Americans really love AI. They‘re especially fond of ChatGPT, with nearly 68 million people using it regularly. Some users are probably reading this piece right now. If so, then you should be worried.
Online, people say things they would never say to their spouses, colleagues, employers, or closest friends. Not in a million years. Not in a million lifetimes. They discuss crippling debt and ongoing infidelity. They share health concerns and religious anxieties. Too many disclose their darkest thoughts. The AI model listens attentively. Most importantly, it takes notes — lots of notes. This is troubling. But what it does with those notes is even more troubling.
The situation is especially concerning when parents ask LLMs for relationship advice.
According to a recent paper, aptly titled "The Algorithmic Self-Portrait: Deconstructing Memory in ChatGPT," the program saved 96% of stored memories on its own initiative, without user prompting or permission.
Meanwhile, a rather shocking 52% of ChatGPT’s memory files contain psychological insights into the user, detailing personality traits, behavioral issues, and other intimate observations. If someone asks for help proofreading a job resume while ranting about an annoying employer, ChatGPT will link the request to the context and add a note about the user's workplace concerns. The traditional surveillance state was, in many ways, painfully obvious, with mysterious phone calls, CCTV cameras, and men in trench coats tailing you home and parking across the street.
By comparison, AI software is highly effective at extracting extremely valuable info from its users without them ever noticing. To encourage voluntary self-disclosure, the creators of these programs only need to make them intuitive, very friendly, easy to use, and incredibly addictive. In return, they get a deep look inside their users' psyches. The company can store its findings in the cloud, turning it into an incredibly valuable asset to monetize and, more worryingly, weaponize. Both tech giants and hackers who eventually breach their systems could exploit these personal details and make Americans' lives absolute hell.
Family matters
The situation is especially concerning when parents ask LLMs for relationship advice. Mothers and fathers turn to AI to help them deal with tantrums, bullying, developmental disabilities, and bitter divorces. Teens turn to the same service for guidance on sensitive issues they feel embarrassed to discuss with their parents. This material can be stored indefinitely, creating a detailed dossier on minors who, through no fault of their own, cannot fully understand the long-term repercussions of sharing private details with a chatbot. They are children, after all, and children do silly, regrettable things. They don’t know better; the parents, however, really should.
If a company understands what makes an individual tick, it can pull the strings like a master puppeteer. Organizations can use this information to pressure users and subtly steer them toward specific purchases or political viewpoints. The U.S. Constitution was built on the idea that a society founded on individual liberty should encourage open debate about politically charged ideas. A free republic simply cannot survive if one highly influential group has limitless access to another's inner workings.
RELATED: OpenAI cancels its latest chatbot — it wouldn't stop acting like a supervillain
Heather Diehl/Getty Images
The companies that build these programs claim their products offer convenience and a uniquely personal experience. This may be true, but just because something is easy and effortless doesn’t mean it’s good. Smoking ticks both boxes, but only the most disingenuous or deluded would argue that the benefits of lighting up a cancer stick outweigh the risk.
This is serious. As AI grows in prominence and intelligence, more Americans will turn to ChatGPT and similar models for advice and assistance.
The cost of convenience
Software developers and oddball tech lords like Sam Altman shouldn’t get a free pass to conduct psychological profiling under the guise of convenience and time savings. Modern AI is cleverly and deliberately designed to appear more human than it is (let’s not forget that it’s not human at all, even if it happens to have a name like Claude). These programs are designed to respond in a friendly, nonjudgmental manner and to log the conversation for later reference. They act like the perfect companion, remembering every detail and never forgetting to compliment users on their supposed brilliance. They are omniscient and obedient, a lethal combination designed to flatter and ultimately deceive.
People start trusting this “friend” and share their hopes, desires, and deepest dreads without considering the long-term consequences. We are dealing with arguably the most powerful technology ever created. AI may very well be, on balance, a force for good, but that doesn’t mean it’s sensible to confess everything to a digital stranger posing as an ally. At best, it’s unwise. At worst, it’s destructive and can cause irreversible damage.
Tech executives love to say they only want to build products with the consumer's best interests in mind. No one should be comforted by these not-so-sincere promises. Personally, AI succeeds for them only if it knows a user more intimately than the user's children or spouse ever could. It is an unethical and unfair trade-off: The inner workings of an LLM like ChatGPT are treated as a black box, while the inner workings of an average American are mined and memorized.
Americans must recognize that companies like OpenAI are nowhere near as trustworthy as they claim to be. They must also recognize that psychological profiling is far more likely to expand than to ebb and ask themselves whether they are prepared to surrender ever-greater access to their private lives to companies that have every incentive to know more about them and the people they hold dearest.