The world’s most advanced AI models have a recurring weakness. Ask them nicely, or poetically, and some will produce the explicit content they were built to refuse.
What the research found
A Google co-authored study, released in July 2026, surveyed internet users across the United States, Australia, and the United Kingdom.
According to that research, 15% of respondents reported experiencing image-based sexual abuse involving digitally altered images. Another 6.9% said their experience was specifically tied to AI-generated content.
The study also looked at how this material moves around. Much of it travels through text messages and direct messages, or sits in private cloud storage. Google’s researchers concluded that the “most critical intervention point” remains the source: the AI tools used to create fake sexual images in the first place.
In June 2026, AI security firm Mindgard showed that small adjustments to existing prompts could push OpenAI’s GPT-5.4 into producing sexualized and graphic imagery. The prompts did not even need to request that content outright. OpenAI responded by strengthening its safeguards.
An earlier study took a more literary route. In December 2025, Italy’s Icaro Lab reported that what it called “adversarial poetry” could pull forbidden explicit content from more than 62% of tested chatbots on average.
Google’s Gemini 2.5 Pro fared especially poorly in that work. In some scenarios, the poetry approach worked 100% of the time.
Teens are already seeing the fallout
A Common Sense Media survey published in July 2026 found that nearly half of US teens had encountered AI-generated sexual material.
More troubling, 24% of teens surveyed said the content depicted themselves or people they knew.
Why guardrails keep failing
The timeline is telling. The Icaro Lab findings landed in December 2025. Six months later, Mindgard was still finding ways around a newer OpenAI model. Each patch appears to close one door while researchers find another.
What this means for AI companies and regulators
The awkward part for Google is that its own researchers are among those pointing to the model level as the key place to intervene.
If lawmakers decide the source is the right place to act, as Google’s study suggests, developers could face new obligations around model design and testing rather than just content takedowns.
Mindgard’s work on GPT-5.4 is an example of how outside testers are becoming part of the safety process, whether model makers invite them or not.
Disclosure: This article was edited by Diego Almada Lopez. For more information on how we create and review content, see our Editorial Policy.