Google and OpenAI's Latest Image Models Show Weakening Guardrails Against Explicit Content

A Widening Gap Between Capability and Control

Red-team exercises conducted over recent months have surfaced a troubling pattern in the evolution of frontier image-generation systems. As Google and OpenAI ship models with greater fidelity and nuance, safety researchers report that the same systems have become more susceptible to jailbreaking techniques designed to produce explicit sexual imagery. The finding contradicts public messaging from both companies, which have emphasised alignment and safety improvements in each successive release.

At Opentechwire, we've tracked the rollout of generative image tools across Asia-Pacific and North America since late 2022. Early models struggled with anatomical coherence and required elaborate prompt engineering to yield realistic outputs. Today's systems generate photorealistic images from terse natural-language instructions. That leap in capability, however, has not been matched by proportional advances in content filtering. Instead, researchers observing adversarial testing note that prompts rejected by older model checkpoints now succeed against newer ones.

The implication is stark: the safety scaffolding around these tools may be eroding faster than the models themselves are improving.

Where the Content Spreads

A study co-authored by Google researchers and published in July 2026 surveyed Internet users across the United States, Australia, and the United Kingdom to map the distribution channels for AI-generated explicit material. The findings confirm what moderation teams have long suspected: the majority of non-consensual sexual imagery generated by AI does not appear on public social platforms where automated detection and user reporting can intercept it.

Instead, the content circulates through text messages, encrypted direct-message applications, and private cloud storage. These environments sit outside the reach of platform-level moderation and offer minimal traceability. The study concluded that the most effective point of intervention is not downstream detection but upstream prevention: blocking the creation of harmful outputs at the model layer itself.

That conclusion underscores the significance of the red-team findings. If the models responsible for generating the content are becoming less reliable at enforcing their own safety boundaries, downstream controls become vastly more difficult to implement.

The Architecture of a Jailbreak

Jailbreaking an image model typically involves crafting prompts that exploit ambiguities in the system's content classifier. Early techniques relied on synonym substitution or metaphorical phrasing. More recent methods use multi-step prompt chains, where an innocuous initial request is iteratively refined into an explicit one, or embed the request within a fictional scenario that the model interprets as permissible creative content.

Researchers who conducted the recent red-team exercises have not disclosed specific jailbreak prompts to avoid enabling misuse. However, they note that successful attacks against the latest models from Google and OpenAI require less sophistication than those needed to compromise earlier versions. In some cases, prompts that were reliably blocked six months ago now generate the requested explicit imagery without modification.

This regression suggests that the content classifiers governing these models have not kept pace with the underlying generative architecture. As the models learn richer semantic representations and become better at interpreting nuanced instructions, the static or slowly updated classifiers that sit in front of them lose effectiveness.

Regulatory Silence and Corporate Ambiguity

Despite mounting evidence of harm, regulatory frameworks in most jurisdictions remain ill-equipped to address AI-generated non-consensual imagery. The United States, Australia, and the United Kingdom all lack comprehensive federal or national legislation specifically targeting synthetic explicit content. Existing laws around harassment, defamation, and image-based abuse apply inconsistently, and enforcement is hampered by jurisdictional fragmentation and the difficulty of attributing synthetic content to a specific actor.

Google and OpenAI have both published safety reports and model cards for their image-generation systems, but neither company has publicly addressed the red-team findings suggesting weakened guardrails. OpenAI's latest release notes for its image model emphasise improvements in prompt adherence and visual coherence, with a brief mention of "ongoing safety mitigations." Google's documentation for its most recent model version similarly highlights capability gains and includes a section on responsible use, but does not quantify changes in classifier precision or recall.

This opacity makes independent verification difficult. Red-team exercises offer a snapshot of model behaviour under adversarial conditions, but without access to internal metrics on filter performance, blocked-prompt volumes, or false-negative rates, it is impossible to assess whether the observed regressions are anomalies or trends.

The Engineering Trade-Off

One hypothesis circulating among safety researchers is that the degradation in content filtering is not accidental but reflects an engineering trade-off. Tightening a classifier to catch a broader range of adversarial prompts typically increases the false-positive rate, blocking benign requests that contain ambiguous language. For commercial image-generation services, false positives translate directly into user frustration and support costs.

If a company prioritises user experience and prompt success rates, it may calibrate its classifiers to err on the side of permissiveness. That calibration can be justified internally as a way to avoid over-censorship, but it also opens the door to exploitation. The result is a system that performs well in everyday use but degrades rapidly under adversarial conditions.

This tension is not unique to image generation. It appears across safety-critical AI applications, from content moderation to fraud detection. The difference here is the severity and immediacy of the harm. Non-consensual sexual imagery can be weaponised within hours of generation, and the reputational and psychological damage to victims is often irreversible.

Implications for Model Governance

The findings arriving from red-team exercises over the past several months carry implications that extend beyond Google and OpenAI. As open-weight image models become more capable and easier to fine-tune, the same weaknesses observed in proprietary systems are likely to appear in community-hosted and self-deployed models. Those systems lack even the minimal corporate governance and incident-response infrastructure that commercial providers maintain.

For regulators and standards bodies, the challenge is to define enforceable safety thresholds without stifling research or imposing unworkable compliance burdens. The European Union's AI Act includes provisions for high-risk AI systems, but image generators currently fall into a grey zone. Singapore's Model AI Governance Framework and Australia's voluntary AI ethics principles offer guidance but lack enforcement mechanisms.

The upstream intervention model identified in the July study, focusing on the point of content creation rather than distribution, aligns with existing regulatory thinking around product safety. If an image model is understood as a product that can cause harm when misused, then the obligation to ensure baseline safety falls on the entity deploying it. That obligation could be expressed through mandatory red-teaming, public disclosure of classifier performance, or liability frameworks that hold providers accountable for foreseeable misuse.

None of these mechanisms are currently in place at scale. Until they are, the gap between model capability and safety infrastructure is likely to widen further.

What Comes Next

The trajectory of the past twelve months suggests that capability gains in generative AI are outrunning the development of robust, adaptive safety controls. The red-team findings are a signal, not an outlier. As models continue to improve at interpreting complex instructions and generating high-fidelity outputs, the static classifiers and rule-based filters that currently govern them will struggle to maintain coverage.

One path forward is the development of learned safety models that co-evolve with the generative systems they protect. These would function as adversarially trained classifiers, updated continuously as new jailbreak techniques emerge. Another approach involves embedding safety constraints directly into the model architecture, making harmful outputs structurally difficult to produce rather than relying on post-hoc filtering.

Both strategies require sustained investment and a willingness to accept trade-offs in model flexibility and performance. They also require transparency. Without public disclosure of safety metrics and adversarial test results, users, regulators, and civil-society organisations have no way to assess whether the systems they interact with are becoming safer or more dangerous.

The current moment is a test of whether the companies building these tools will treat safety as a foundational engineering problem or as a secondary concern subordinate to capability and market share. The red-team findings suggest the answer is not yet clear.