The safety benchmarks enterprise buyers rely on to evaluate AI models are measuring the wrong thing. That’s the finding from recent Cisco research pairing single-turn and multi-turn evaluations across 15 closed frontier models from OpenAI, Anthropic, Google, Amazon, and xAI. Every model failed a non-trivial share of multi-turn attacks, and the success rates of those attacks ranged from 7.89% to 88.30% across the cohort — a wider spread than the single-turn range of 2.19% to 64.91%. Single-turn is a one-and-done interaction. Multi-turn is a continuous back-and-forth dialogue. “Multi-turn evaluation matters for one primary reason: it is where attackers operate,” the report states. “Real adversaries iterate, reframe refusals, decompose tasks across turns, adopt personas, and escalate gradually.”

Full report : Cisco tested 15 flagship AI models and found that the safety benchmarks guiding enterprise purchases consistently understate how those models break down under sustained, multi-turn attacks.