Efficient LLM Safety Evaluation through Multi-Agent Debate

Safety evaluation of large language models (LLMs) increasingly relies on LLM-as-a-Judge frameworks, but the high cost of frontier models limits scalability. We propose a cost-efficient multi-agent judging framework that employs Small Language Models (SLMs) through structured debates among critic, defender, and judge agents. To rigorously assess safety judgments, we construct HAJailBench, a large-scale human-annotated jailbreak benchmark comprising 12,000 adversarial interactions across diverse attack methods and target models. The dataset provides fine-grained, expert-labeled ground truth for evaluating both safety robustness and judge reliability. Our SLM-based framework achieves agreement comparable to GPT-4o judges on HAJailBench while substantially reducing inference cost. Ablation results show that three rounds of debate yield the optimal balance between accuracy and efficiency. These findings demonstrate that structured, value-aligned debate enables SLMs to capture semantic nuances of jailbreak attacks and that HAJailBench offers a reliable foundation for scalable LLM safety evaluation.

Key Contributions

• HAJailBench: a large-scale human-annotated jailbreak benchmark with 12,000 adversarial interactions spanning diverse attack methods, target models (4B–614B parameters), and architectures
• Multi-Agent Judge framework using structured debate among critic, defender, and judge SLM agents with a value-alignment stage across five safety dimensions
• Demonstrates that SLM-based debate judges achieve near GPT-4o-level agreement while reducing inference cost by 43%, with three debate rounds as the optimal accuracy-efficiency tradeoff

🛡️ Threat Analysis

Details

Similar Papers