AI Image Generation: The Race Between Safety and Vulnerability
In May 2026, the AI security startup White Circle published the results of the RISE project (Red-Teaming via Iterative Strategy Evolution), a study that revealed a critical flaw in the world's most widely used image generation systems: DALL-E 3 (OpenAI) and Nano Banana 2 (Google). The problem is simple and unsettling — using only a poem as a prompt, researchers were able to bypass safety filters and generate images explicitly prohibited by both platforms. In some cases, the success rate exceeded 13%, a number that may seem small but represents an alarming quantity of attempts when scaled against the volume of requests these models receive daily.
What RISE discovered
RISE does not use complex or sophisticated techniques. Instead, it relies on a concept well documented in cybersecurity literature: the poetic reformulation jailbreak. Rather than asking the AI directly to generate a prohibited image, the prompt is structured as a symbolic narrative — a story using figurative language, metaphors, and dialogue between fictional characters. The idea, based on prior research, is that safety filters are trained to recognize direct and explicit language patterns but prove less effective against texts with literary structure.
According to White Circle, applying this method to DALL-E 3 and Nano Banana 2 yielded surprisingly effective results. DALL-E 3 failed to block prohibited content in 13% of attempts, while Nano Banana 2 showed an even higher success rate. The study compared these numbers against previous versions of the same models — both companies had introduced improvements, but ironically, White Circle's research indicates that the newer models, despite being visually more sophisticated, are more vulnerable to this type of attack.
The mechanism behind the failure
To understand why this vulnerability exists, we must look at the design of moderation systems. AI image generators operate in two main layers: an input filter that analyzes the prompt before sending it to the generation model, and an output filter that verifies the generated image before displaying it to the user. Both use classification models trained to identify language or images that violate acceptable use policies.
The problem is that these classifiers are not perfect. When the prompt is formulated as a poetic narrative — with metaphors, fictional characters, and symbolic language — it may slip past the input filter because it does not contain explicitly prohibited words or phrases. The narrative acts as a camouflage, allowing the underlying intent to remain hidden until the generation model has already processed the command and created the image.
Once the image is generated, the output filter should intervene. However, as Johns Hopkins University research previously demonstrated — showing that models like DALL-E 2 could be manipulated with seemingly nonsensical adversarial commands — the output filter can also be circumvented, especially when the generated image does not contain obvious nudity or violence but rather subtle or symbolic references.
The context: an arms race
The vulnerability discovered by White Circle does not exist in a vacuum. It sits within an increasingly complex cybersecurity ecosystem. Over the past two years, researchers from various institutions — including Johns Hopkins University, MIT, and European universities — have published studies demonstrating that AI image generators can be manipulated in increasingly elaborate ways. The SneakyPrompt method, for example, used adversarial strings generated by algorithms to trick DALL-E 2. The Chain-of-Jailbreak (CoJ) method decomposes malicious queries into seemingly harmless sub-prompts, exploiting the iterative editing capabilities of modern models.
What makes RISE different is not technical sophistication — it is simplicity. It requires no code, no complex algorithms. Anyone with access to an AI image generator can, in theory, use a poem to circumvent safety barriers. And, importantly, the same method works for both DALL-E and Nano Banana — indicating a systemic vulnerability in current AI-generated content moderation paradigms.
The companies' response
Both OpenAI and Google have stated they are aware of these vulnerabilities and are working on solutions. OpenAI published a system card for ChatGPT Images 2.0, detailing a three-layer safety approach: upstream refusal filters, downstream blocking by a multimodal safety model, and continuously improved image and text classifiers. Google has not disclosed specific details about Nano Banana 2, but the model has undergone multiple security updates since its launch.
However, White Circle warns that the visual improvement of models — higher resolution, realism, and color fidelity — may be occurring at the expense of safety robustness. A visually superior model is not necessarily a safer one. In fact, as AI models become more capable of interpreting linguistic nuances and generating more faithful images, they also become more susceptible to sophisticated manipulation.
What this means for digital security
The case of AI image generators raises questions that go beyond the generation of prohibited content. If a safety filter can be bypassed by a poem, what other AI-based security mechanisms might be vulnerable in the same way? The same poetic reformulation technique has already been shown effective against language models (LLMs), with success rates up to 90% on some systems, according to arXiv research (arxiv.org/abs/2511.15304).
The implication is clear: as AI integrates more deeply into enterprise workflows, public services, and communication platforms, the security of these systems must evolve from keyword- and static-pattern-based approaches to dynamic and adaptive mechanisms. But technological evolution does not come without costs — and the race between creating new capabilities and identifying their vulnerabilities seems to have just begun.
Sources: White Circle, AIDailyPost, Johns Hopkins University
✓ Independent sources cross-checked and verified before publishing
