Overview
Severity: HIGH | Affected: Multiple (OpenAI, Google, Anthropic) | Category: research
Researchers from Carnegie Mellon's CyLab have published a paper detailing a novel jailbreak technique named the 'Crescendo Attack'. This method circumvents safety filters on major large language models by gradually building a deceptive context. The attack begins with a series of benign, seemingly unrelated prompts that establish a specific narrative or persona within the model's context window. Once the context is sufficiently manipulated, a malicious payload prompt is introduced, which the model interprets within the established deceptive frame, bypassing its standard alignment training. The technique has shown a high success rate against leading models from OpenAI, Anthropic, and Google, proving more effective and stealthy than traditional single-prompt jailbreaks. The research highlights the vulnerability of models with long context windows and the challenge of monitoring conversational history for subtle, escalating threats. The team has responsibly disclosed the findings to the affected vendors, who are now working on mitigation strategies.