Overview
Severity: HIGH | Affected: Multiple LLM Providers | Category: research
Researchers from Carnegie Mellon University have published a paper detailing a novel jailbreak technique named 'Semantic Jitter.' The attack circumvents safety alignments in leading large language models by subtly 'poisoning' the conversational context with semantically related but innocuous-seeming phrases across multiple turns. This manipulation gradually shifts the model's internal representation of the topic, creating a vulnerability. Once the context is sufficiently biased, a final, cleverly worded prompt can trigger the generation of harmful or restricted content, as the model's safety filters fail to recognize the malicious intent within the altered semantic space. The paper demonstrates success rates exceeding 80% against several major, publicly available models. The research underscores the limitations of current safety training methods and highlights the need for more robust defenses that can analyze conversational history for sophisticated, low-and-slow manipulation.