ML Security PapersAdvEvo-MARL: Shaping Internalized Safety through Adversarial Co-Evolution in Multi-Agent Reinforcement Learning
Zhenyu Pan 1, Yiting Zhang 2, Zhuo Liu 3, Yolo Yunlong Tang 3, Zeliang Zhang 3, Haozheng Luo 1, Yuwei Han 2, Jianshu Zhang 1, Dennis Wu 1, Hong-Yu Chen 1, Haoran Lu 1, Haoyang Fang 4, Manling Li 1, Chenliang Xu 3, Philip S. Yu 2, Han Liu 1
Published on arXiv
2510.01586
Prompt Injection
OWASP LLM Top 10 — LLM01
Excessive Agency
OWASP LLM Top 10 — LLM08
Key Finding
AdvEvo-MARL keeps attack-success rate below 20% across all tested scenarios while baselines reach up to 38.33%, and improves task accuracy by up to 3.67% on reasoning benchmarks.
AdvEvo-MARL
Novel technique introduced
LLM-based multi-agent systems excel at planning, tool use, and role coordination, but their openness and interaction complexity also expose them to jailbreak, prompt-injection, and adversarial collaboration. Existing defenses fall into two lines: (i) self-verification that asks each agent to pre-filter unsafe instructions before execution, and (ii) external guard modules that police behaviors. The former often underperforms because a standalone agent lacks sufficient capacity to detect cross-agent unsafe chains and delegation-induced risks; the latter increases system overhead and creates a single-point-of-failure-once compromised, system-wide safety collapses, and adding more guards worsens cost and complexity. To solve these challenges, we propose AdvEvo-MARL, a co-evolutionary multi-agent reinforcement learning framework that internalizes safety into task agents. Rather than relying on external guards, AdvEvo-MARL jointly optimizes attackers (which synthesize evolving jailbreak prompts) and defenders (task agents trained to both accomplish their duties and resist attacks) in adversarial learning environments. To stabilize learning and foster cooperation, we introduce a public baseline for advantage estimation: agents within the same functional group share a group-level mean-return baseline, enabling lower-variance updates and stronger intra-group coordination. Across representative attack scenarios, AdvEvo-MARL consistently keeps attack-success rate (ASR) below 20%, whereas baselines reach up to 38.33%, while preserving-and sometimes improving-task accuracy (up to +3.67% on reasoning tasks). These results show that safety and utility can be jointly improved without relying on extra guard agents or added system overhead.
Key Contributions
- AdvEvo-MARL: a co-evolutionary MARL framework that jointly trains attacker agents (synthesizing evolving jailbreak prompts) and defender agents (task agents that resist attacks), internalizing safety without external guard modules.
- Public baseline mechanism for group-level advantage estimation, where agents within the same functional group share the group's mean return as a baseline to reduce variance and strengthen intra-group coordination.
- Empirical demonstration across three MAS attack scenarios (agent manipulation, message corruption, user instruction hijacking) keeping ASR below 20% vs. baselines up to 38.33%, with up to +3.67% task accuracy gain.