Do you remember those 200 questions you asked ChatGPT — the ones you’d be far too embarrassed to ask your own mom?
You may have thought it was all confidential, but a growing number of cases show these chats are not protected by any legal safeguards and could end up in court.
In May 2025, a US court ordered OpenAI to hand over more than 20 million ChatGPT logs that users had previously deleted in connection with the New York Times copyright case.
And just two months ago, a Washington Post investigation revealed that at least a dozen court cases had cited logs from AI chatbots as evidence, highlighting that users’ most intimate confessions could be used for legal purposes.
In US v. Heppner, Judge Jed Rakoff ruled that a defendant's Claude conversations weren't shielded by attorney-client privilege or work-product protection, reasoning that no attorney-client relationship exists "or could exist, between an AI user and a platform such as Claude."
"Most people treat an AI like a private diary. But legally, it’s a record held by a company, with none of the privilege that protects what you tell a lawyer or a doctor," Eamonn Maguire, Director of Engineering and Machine Learning at Proton — which develops its own privacy-focused AI assistant — tells TechRadar.
Despite this, people continue to ignore the weight that their digital diaries carry in the eyes of the law.
When an AI chatbot knows too much
A recent DuckDuckGo survey found that as many as 75 percent of respondents were unaware that their AI chatbot conversations could be subpoenaed by law enforcement.
"We see it with traditional search too," says DuckDuckGo’s SVP of Insights Zac Pappis. "Google, for example, received tens of thousands of search warrants, subpoenas, and other requests for user data, and in most cases, they provide it."
Whether AI chats are used in front of a judge depends on the case, and privacy regulations around the world remain a complex patchwork of national laws, international frameworks, and legal custom.
In the UK, there are no specific laws regulating how AI firms hand over their data.
"The UK has not yet developed a bespoke regulatory regime governing AI or AI conversations," notes Melissa Hall, Legal Director and AI Specialist at Scottish law firm Morton Fraser MacRoberts.
AI providers are also actively scanning chats and referring people to the police themselves. This month, a Florida woman was charged after Anthropic’s system flagged her Claude conversations for human review, prompting the company to contact the police. "She reportedly used it like a diary, unaware staff could read it," Maguire notes.
Unsurprisingly, these systems can also make incorrect legal assessments. Anthropic, for example, recently disclosed that one of its own AI agents submitted a fabricated witness tip to a police murder-tip portal during testing.
"As agents act more and more on our behalf, it’s easy to imagine one misreading a conversation and filing a report on its own or being deliberately directed to make a false report about someone, at scale," Maguire adds.
Even if you try to obscure details, you aren't necessarily safe. As Phil James, Partner and Head of Data, Privacy and Cybersecurity at law firm Browne Jacobson, warns: "Just because you redact someone’s name doesn’t mean they’re not identifiable."
Treating a chatbot like a diary often also means venting about other people. "People forget that personal information about third parties can also be protected by data-protection laws," James stresses. "You’re sharing third-party information which you didn’t have consent to share."
Deleting won’t help
Hit ‘Delete’ as many times as you like: when a court orders data to be retained, your digital footprint remains there.
Venetia Argyropoulou, General Counsel at ExpressVPN, which recently built its own privacy-first LLM, notes: "Most people assume that when they delete a chat, it’s gone, while in reality, it can be kept for some time afterward. Indeed, privacy policies tend to say data can be held for longer if the law requires it."
Under Article 17 of the UK GDPR, users may request the erasure of their personal data, but there are exceptions. "If an organization is required by law to retain certain information, including for the purpose of preserving evidence relating to legal proceedings, such obligations will generally take precedence over a request for erasure," says Hall.
Some might argue that Big Tech companies should simply make their privacy policies clearer. "People deserve a clear warning before they start typing, not one buried deep in a policy document," says Proton's Maguire.
In the UK, the Information Commissioner's Office guidance rules that privacy information must be easily accessible and written in clear and plain language, so simply having a privacy policy hidden on a website may no longer be enough.
"It is possible that, if the use of chat data becomes a higher-risk area from a regulatory perspective, expectations around the prominence of such disclosures may evolve," Hall explains.
NEW: Leave No Trace — A weekly newsletter on digital privacy and online surveillance.
Leave No Trace investigates the companies and governments putting our digital freedom at risk — and the people fighting back.
📩 Subscribe now to get every edition delivered to your inbox every Friday, launching this September.
Safer ground
While we wait for that to happen, companies developing alternative, 'privacy-first' AI models argue that the architecture itself needs to change.
Proton is clear on how to protect user data: "The only real protection is a provider that never holds your conversations in a readable form: you can't hand over what you can't read," says Maguire.
ExpressVPN agrees. "Companies can challenge these requests, but if a court orders them to preserve or produce data, they have to comply. Users have very little say in it, and in most cases won't know it's happening," Argyropoulou says.
In short, if you wouldn't be comfortable with something you've typed into a chatbot ending up in a legal dispute, it's worth thinking carefully about which AI tool you use, she says.
Argyropoulou recommends considering AI tools in which conversations are end-to-end encrypted right up to the user’s browser and protected within a secure enclave, where transcripts and logs cannot be viewed, so that the company cannot provide them even if it wanted to.
"As people trust AI with more of their lives, privacy has to be built into how these tools work, not just written into a policy," Argyropoulou concludes.