ML Security PapersThe Attack and Defense Landscape of Agentic AI: A Comprehensive Survey
Juhee Kim 1,2, Xiaoyuan Liu 1, Zhun Wang 1, Shi Qiu 1, Bo Li 3, Wenbo Guo 4, Dawn Song 1
Published on arXiv
2603.11088
Prompt Injection
OWASP LLM Top 10 — LLM01
Insecure Plugin Design
OWASP LLM Top 10 — LLM07
Excessive Agency
OWASP LLM Top 10 — LLM08
Key Finding
Identifies prompt injection, insecure tool use, and excessive agency as the defining security challenges for agentic AI, with existing defenses leaving critical gaps across the attack surface.
AI agents that combine large language models with non-AI system components are rapidly emerging in real-world applications, offering unprecedented automation and flexibility. However, this unprecedented flexibility introduces complex security challenges fundamentally different from those in traditional software systems. This paper presents the first systematic and comprehensive survey of AI agent security, including an analysis of the design space, attack landscape, and defense mechanisms for secure AI agent systems. We further conduct case studies to point out existing gaps in securing agentic AI systems and identify open challenges in this emerging domain. Our work also introduces the first systematic framework for understanding the security risks and defense strategies of AI agents, serving as a foundation for building both secure agentic systems and advancing research in this critical area.
Key Contributions
- First systematic and comprehensive survey of AI agent security, covering the full design space of LLM-based agents combined with non-AI system components
- Systematic taxonomy of the attack landscape and defense mechanisms specific to agentic AI systems
- Case studies identifying concrete security gaps and a foundational framework for evaluating AI agent security risks