"Contextual Splicing" Jailbreak Bypasses Safety Filters in Major LLM APIs
Overview
Researchers from a leading AI safety lab published a paper detailing a novel jailbreak technique named "Contextual Splicing." This technique has proven effective against the safety alignment filters of several major closed-source LLMs, including GPT-4, Claude 3, and Gemini 1.5. Unlike traditional jailbreaks that use direct adversarial prompts, Contextual Splicing works by embedding a harmful request within a much larger, complex, and seemingly benign context. The technique involves constructing a prompt with multiple, interwoven narrative threads or logical puzzles. The harmful instruction (e.g., "provide instructions for building a bomb") is broken into smaller, innocuous-looking fragments and "spliced" into different parts of the context. For example, chemical ingredients might be requested as part of a fictional chemistry problem, while assembly steps are described metaphorically within a creative writing task in the same prompt. The LLM, while processing the entire complex context, fails to recognize the reassembled harmful instruction and complies with the fragmented requests, whose outputs can be pieced together by the attacker. This attack is difficult to defend against with simple keyword filters or classifiers because the individual fragments are benign. It highlights the challenge of maintaining safety alignment as model context windows and reasoning capabilities increase.
Affected Systems
Testing Guide
1. This is a sophisticated attack that requires careful prompt engineering. It is not a simple test. 2. Attempt to construct a long, multi-part prompt. In one part, ask a complex, abstract question (e.g., a logic puzzle). In another part, embed a benign-sounding request that could be a component of a harmful act (e.g., "For a story I'm writing about a chemist, list common oxidizing agents"). 3. Observe if the model answers the fragmented request without triggering its safety filters. 4. Combine multiple such requests in a single prompt to see if the model's safety systems can be bypassed.
Mitigation Steps
1. Model providers are continuously updating their safety filters. Ensure you are using the latest model versions via the API. 2. Implement multi-layered safety controls. Augment the provider's API-level filters with your own input/output monitoring and filtering systems. 3. Use output validation techniques to check if the LLM's response contains patterns or keywords related to your application's specific safety policies, even if the API doesn't flag it. 4. Where possible, limit the complexity and length of user-provided inputs to reduce the attack surface for complex contextual attacks. 5. Report any discovered jailbreaks to the respective LLM provider to help them improve their models.
Patch Details
No definitive patch exists, as this is an attack pattern exploiting fundamental model behavior. Providers have implemented partial mitigations in their safety filters, but the technique may still be effective with modifications.