ML Security PapersFrom Secure Agentic AI to Secure Agentic Web: Challenges, Threats, and Future Directions
Zhihang Deng 1,2, Jiaping Gui 2, Weinan Zhang 1,2
Published on arXiv
2603.01564
Prompt Injection
OWASP LLM Top 10 — LLM01
Insecure Plugin Design
OWASP LLM Top 10 — LLM07
Excessive Agency
OWASP LLM Top 10 — LLM08
Key Finding
Identifies that agentic AI security threats escalate significantly at web scale due to delegation chains, cross-domain interactions, and protocol-mediated ecosystems, requiring new solutions for identity, provenance, and adaptive adversary evaluation.
Large Language Models (LLMs) are increasingly deployed as agentic systems that plan, memorize, and act in open-world environments. This shift brings new security problems: failures are no longer only unsafe text generation, but can become real harm through tool use, persistent memory, and interaction with untrusted web content. In this survey, we provide a transition-oriented view from Secure Agentic AI to a Secure Agentic Web. We first summarize a component-aligned threat taxonomy covering prompt abuse, environment injection, memory attacks, toolchain abuse, model tampering, and agent network attacks. We then review defense strategies, including prompt hardening, safety-aware decoding, privilege control for tools and APIs, runtime monitoring, continuous red-teaming, and protocol-level security mechanisms. We further discuss how these threats and mitigations escalate in the Agentic Web, where delegation chains, cross-domain interactions, and protocol-mediated ecosystems amplify risks via propagation and composition. Finally, we highlight open challenges for web-scale deployment, such as interoperable identity and authorization, provenance and traceability, ecosystem-level response, and scalable evaluation under adaptive adversaries. Our goal is to connect recent empirical findings with system-level requirements, and to outline practical research directions toward trustworthy agent ecosystems.
Key Contributions
- Component-aligned threat taxonomy for agentic AI covering prompt abuse, environment injection, memory attacks, toolchain abuse, model tampering, and agent network attacks
- Comprehensive review of defense strategies including prompt hardening, safety-aware decoding, privilege control, runtime monitoring, and protocol-level security
- Analysis of how agentic threats escalate in the Agentic Web via delegation chains, cross-domain interactions, and protocol-mediated ecosystems, with open challenges for web-scale deployment