ML Security PapersCan an Individual Manipulate the Collective Decisions of Multi-Agents?
Fengyuan Liu 1,2, Rui Zhao 1, Shuo Chen 3,4,5, Guohao Li 2, Philip Torr 2, Lei Han 1, Jindong Gu 2
Published on arXiv
2509.16494
Input Manipulation Attack
OWASP ML Top 10 — ML01
Prompt Injection
OWASP LLM Top 10 — LLM01
Key Finding
M-Spoiler successfully misleads collective multi-agent LLM decisions using adversarial suffixes generated from only a single known agent, remaining more effective than baselines even under multiple defense mechanisms across 9 models and 7 datasets.
M-Spoiler
Novel technique introduced
Individual Large Language Models (LLMs) have demonstrated significant capabilities across various domains, such as healthcare and law. Recent studies also show that coordinated multi-agent systems exhibit enhanced decision-making and reasoning abilities through collaboration. However, due to the vulnerabilities of individual LLMs and the difficulty of accessing all agents in a multi-agent system, a key question arises: If attackers only know one agent, could they still generate adversarial samples capable of misleading the collective decision? To explore this question, we formulate it as a game with incomplete information, where attackers know only one target agent and lack knowledge of the other agents in the system. With this formulation, we propose M-Spoiler, a framework that simulates agent interactions within a multi-agent system to generate adversarial samples. These samples are then used to manipulate the target agent in the target system, misleading the system's collaborative decision-making process. More specifically, M-Spoiler introduces a stubborn agent that actively aids in optimizing adversarial samples by simulating potential stubborn responses from agents in the target system. This enhances the effectiveness of the generated adversarial samples in misleading the system. Through extensive experiments across various tasks, our findings confirm the risks posed by the knowledge of an individual agent in multi-agent systems and demonstrate the effectiveness of our framework. We also explore several defense mechanisms, showing that our proposed attack framework remains more potent than baselines, underscoring the need for further research into defensive strategies.
Key Contributions
- Frames multi-agent LLM manipulation as an incomplete-information game where the adversary has white-box access to one agent but no knowledge of the others
- Proposes M-Spoiler, introducing a simulated stubborn agent and critical agent to optimize adversarial suffixes that transfer across the full multi-agent system
- Demonstrates empirically across 9 LLMs and 7 datasets that single-agent knowledge is sufficient to compromise collective multi-agent decisions, and that M-Spoiler outperforms baselines under several defense strategies
🛡️ Threat Analysis
M-Spoiler generates gradient-optimized adversarial token suffixes (GCG-style) targeting LLMs at inference time to cause misclassification and misleading outputs — a direct adversarial suffix attack evaluated on AdvBench and classification tasks.