ML Security PapersLet the Bees Find the Weak Spots: A Path Planning Perspective on Multi-Turn Jailbreak Attacks against LLMs
Yize Liu , Yunyun Hou , Aina Sui
Published on arXiv
2511.03271
Prompt Injection
OWASP LLM Top 10 — LLM01
Key Finding
ABC achieves attack success rates above 90% across all five tested LLMs (peak 98% on GPT-3.5-Turbo) using only 26 average queries, outperforming existing multi-turn jailbreak baselines
ABC (Artificial Bee Colony jailbreak)
Novel technique introduced
Large Language Models (LLMs) have been widely deployed across various applications, yet their potential security and ethical risks have raised increasing concerns. Existing research employs red teaming evaluations, utilizing multi-turn jailbreaks to identify potential vulnerabilities in LLMs. However, these approaches often lack exploration of successful dialogue trajectories within the attack space, and they tend to overlook the considerable overhead associated with the attack process. To address these limitations, this paper first introduces a theoretical model based on dynamically weighted graph topology, abstracting the multi-turn attack process as a path planning problem. Based on this framework, we propose ABC, an enhanced Artificial Bee Colony algorithm for multi-turn jailbreaks, featuring a collaborative search mechanism with employed, onlooker, and scout bees. This algorithm significantly improves the efficiency of optimal attack path search while substantially reducing the average number of queries required. Empirical evaluations on three open-source and two proprietary language models demonstrate the effectiveness of our approach, achieving attack success rates above 90\% across the board, with a peak of 98\% on GPT-3.5-Turbo, and outperforming existing baselines. Furthermore, it achieves comparable success with only 26 queries on average, significantly reducing red teaming overhead and highlighting its superior efficiency.
Key Contributions
- Formalizes multi-turn jailbreak attack space as a path planning problem on a dynamically weighted directed graph, where nodes are attack strategies and edge weights reflect sequential effectiveness
- Proposes ABC, an enhanced Artificial Bee Colony algorithm with employed/onlooker/scout bee roles that collaboratively explore and exploit the attack space to find high-success-rate paths efficiently
- Achieves >90% ASR on all five tested LLMs (peak 98% on GPT-3.5-Turbo) while requiring only ~26 queries on average, significantly outperforming existing multi-turn baselines