ML Security PapersAEGIS : Automated Co-Evolutionary Framework for Guarding Prompt Injections Schema
Ting-Chun Liu , Ching-Yu Hsu , Kuan-Yi Lee , Chi-An Fu , Hung-yi Lee
Published on arXiv
2509.00088
Prompt Injection
OWASP LLM Top 10 — LLM01
Key Finding
Co-evolutionary defense achieves a true positive rate of 0.84 (improvement of 0.23 over prior best) with TNR of 0.89, while the co-evolved attacker reaches ASR of 1.0 (improvement of 0.26 over baseline).
AEGIS / TGO+
Novel technique introduced
Prompt injection attacks pose a significant challenge to the safe deployment of Large Language Models (LLMs) in real-world applications. While prompt-based detection offers a lightweight and interpretable defense strategy, its effectiveness has been hindered by the need for manual prompt engineering. To address this issue, we propose AEGIS , an Automated co-Evolutionary framework for Guarding prompt Injections Schema. Both attack and defense prompts are iteratively optimized against each other using a gradient-like natural language prompt optimization technique. This framework enables both attackers and defenders to autonomously evolve via a Textual Gradient Optimization (TGO) module, leveraging feedback from an LLM-guided evaluation loop. We evaluate our system on a real-world assignment grading dataset of prompt injection attacks and demonstrate that our method consistently outperforms existing baselines, achieving superior robustness in both attack success and detection. Specifically, the attack success rate (ASR) reaches 1.0, representing an improvement of 0.26 over the baseline. For detection, the true positive rate (TPR) improves by 0.23 compared to the previous best work, reaching 0.84, and the true negative rate (TNR) remains comparable at 0.89. Ablation studies confirm the importance of co-evolution, gradient buffering, and multi-objective optimization. We also confirm that this framework is effective in different LLMs. Our results highlight the promise of adversarial training as a scalable and effective approach for guarding prompt injections.
Key Contributions
- AEGIS: a GAN-inspired co-evolutionary framework that jointly and iteratively refines both attack and defense prompts against each other without model fine-tuning
- TGO+: an enhanced textual gradient optimization module using multi-route natural language feedback, a gradient buffer, and multi-objective signals to simulate gradient-like prompt updates for black-box LLMs
- Evaluation on a real-world assignment grading dataset showing TPR improvement of 0.20–0.23 over prior SOTA for prompt injection detection while maintaining comparable TNR