Jailbreak-Zero: A Path to Pareto Optimal Red Teaming for Large Language Models

This paper introduces Jailbreak-Zero, a novel red teaming methodology that shifts the paradigm of Large Language Model (LLM) safety evaluation from a constrained example-based approach to a more expansive and effective policy-based framework. By leveraging an attack LLM to generate a high volume of diverse adversarial prompts and then fine-tuning this attack model with a preference dataset, Jailbreak-Zero achieves Pareto optimality across the crucial objectives of policy coverage, attack strategy diversity, and prompt fidelity to real user inputs. The empirical evidence demonstrates the superiority of this method, showcasing significantly higher attack success rates against both open-source and proprietary models like GPT-40 and Claude 3.5 when compared to existing state-of-the-art techniques. Crucially, Jailbreak-Zero accomplishes this while producing human-readable and effective adversarial prompts with minimal need for human intervention, thereby presenting a more scalable and comprehensive solution for identifying and mitigating the safety vulnerabilities of LLMs.

Key Contributions

• Policy-based red teaming framework that fine-tunes an attack LLM with preference data to generate high-volume, diverse adversarial prompts, achieving Pareto optimality across policy coverage, diversity, and prompt fidelity.
• Demonstrates significantly higher attack success rates against proprietary models (GPT-4o, Claude 3.5) compared to existing state-of-the-art red teaming methods.
• Produces human-readable adversarial prompts with minimal human intervention, enabling scalable, automated safety vulnerability discovery.

🛡️ Threat Analysis

Details

Similar Papers