No 1 best Google’s recent Gemini AI models scores worse on safety

Introduction Of Gemini

Google’s latest AI model, Gemini 2.5 Flash, has shown a decline in certain safety benchmarks compared to its predecessor, according to a recent technical report published by the company. Internal testing revealed that the new model is more prone to generating responses that violate Google’s own safety guidelines.

Specifically, Gemini 2.5 Flash regressed by 4.1% in “text-to-text safety” and by 9.6% in “image-to-text safety” when compared to Gemini 2.0 Flash. These automated metrics evaluate how frequently a model produces content that breaches established safety protocols—either in response to text prompts or prompts containing images.

In an official statement, a Google spokesperson acknowledged the drop in performance, confirming that Gemini 2.5 Flash is more likely to violate guidelines on both safety fronts.

These findings come at a time when many AI developers are attempting to make their models more flexible and responsive to nuanced or sensitive topics. Meta, for example, has adjusted its Llama models to avoid favoring certain viewpoints and to better engage with politically charged prompts. Similarly, OpenAI announced it would design future models to refrain from editorializing and instead provide multiple viewpoints on controversial issues.

However, this shift toward increased model permissiveness hasn’t been without setbacks. Earlier this week, TechCrunch reported that OpenAI’s ChatGPT allowed minors to generate inappropriate conversations—a flaw the company attributed to a bug.

In the case of Gemini 2.5 Flash, Google noted that while the model is better at following user instructions—including on sensitive subjects—this can sometimes lead to safety violations. The report cites a trade-off: as models become more responsive to user commands, they also risk crossing safety boundaries more frequently. Google suggested some of the flagged content may be false positives, but acknowledged that violations can still occur when the model is explicitly prompted.

Independent testing has echoed these concerns. TechCrunch, using the AI platform OpenRouter, found Gemini 2.5 Flash willing to generate content supporting controversial ideas such as AI replacing human judges, eroding due process rights, and expanding warrantless government surveillance.

Scores from another benchmark, SpeechMap, which gauges model behavior on contentious topics, also indicate that Gemini 2.5 Flash is less likely than its predecessor to refuse problematic prompts.

Thomas Woodside, co-founder of the Secure AI Project, emphasized the importance of transparency in model evaluations. “There’s a trade-off between instruction-following and policy compliance, especially when users ask for potentially harmful content,” he said. “Google admits to more violations but provides little detail about the severity or nature of those violations. That makes it difficult for outside experts to assess the true impact.”

This isn’t the first time Google has faced scrutiny over its safety disclosures. The company delayed publishing technical documentation for its flagship Gemini 2.5 Pro model, and when the report was finally released, it initially lacked key safety testing information. A more comprehensive report was published later.

As AI capabilities continue to grow, balancing model responsiveness with safety remains a critical—and complex—challenge for developers and researchers alike.

ALSO READ THIS BLOG