ML Security PapersFrom Spark to Fire: Modeling and Mitigating Error Cascades in LLM-Based Multi-Agent Collaboration
Yizhe Xie 1,2, Congcong Zhu 1, Xinyue Zhang 1,2, Tianqing Zhu 1, Dayong Ye 1, Minfeng Qi 1, Huajie Chen 1, Wanlei Zhou 1
Published on arXiv
2603.04474
Prompt Injection
OWASP LLM Top 10 — LLM01
Excessive Agency
OWASP LLM Top 10 — LLM08
Key Finding
Genealogy-graph governance layer raises defense success rate from 0.32 to over 0.89 across six mainstream LLM-MAS frameworks; ablations show detection alone is insufficient without rollback or isolation
genealogy-graph-based governance layer
Novel technique introduced
Large Language Model-based Multi-Agent Systems (LLM-MAS) are increasingly applied to complex collaborative scenarios. However, their collaborative mechanisms may cause minor inaccuracies to gradually solidify into system-level false consensus through iteration. Such risks are difficult to trace since errors can propagate and amplify through message dependencies. Existing protections often rely on single-agent validation or require modifications to the collaboration architecture, which can weaken effective information flow and may not align with natural collaboration processes in real tasks. To address this, we propose a propagation dynamics model tailored for LLM-MAS that abstracts collaboration as a directed dependency graph and provides an early-stage risk criterion to characterize amplification risk. Through experiments on six mainstream frameworks, we identify three vulnerability classes: cascade amplification, topological sensitivity, and consensus inertia. We further instantiate an attack where injecting just a single atomic error seed leads to widespread failure. In response, we introduce a genealogy-graph-based governance layer, implemented as a message-layer plugin, that suppresses both endogenous and exogenous error amplification without altering the collaboration architecture. Experiments show that this approach raises the defense success rate from a baseline of 0.32 to over 0.89 and significantly mitigates the cascading spread of minor errors.
Key Contributions
- IBMF-based propagation dynamics model that abstracts LLM-MAS collaboration as a directed dependency graph and derives the early amplification criterion β·ρ(A) > δ to quantify cascade risk
- Attack instantiation demonstrating that a single role-consistent, intent-hiding error seed injection causes widespread false consensus across six mainstream multi-agent frameworks, identifying vulnerability classes of cascade amplification, topological sensitivity, and consensus inertia
- Genealogy-graph-based governance layer implemented as a message-layer plugin providing claim provenance tracking, targeted verification, and enforced rollback, raising defense success rate from 0.32 to over 0.89 without altering the collaboration architecture