ML Security PapersFrom Insight to Exploit: Leveraging LLM Collaboration for Adaptive Adversarial Text Generation
Najrin Sultana 1, Md Rafi Ur Rashid 1, Kang Gu 2, Shagufta Mehnaz 1
Published on arXiv
2511.03128
Prompt Injection
OWASP LLM Top 10 — LLM01
Key Finding
LLM-guided adversarial perturbations outperform static baselines (PromptAttack, CombinedAttack) and transfer effectively to unseen LLMs, demonstrating systematic robustness weaknesses in zero-shot LLM classifiers.
StaDec / DyDec (Static Deceptor / Dynamic Deceptor)
Novel technique introduced
LLMs can provide substantial zero-shot performance on diverse tasks using a simple task prompt, eliminating the need for training or fine-tuning. However, when applying these models to sensitive tasks, it is crucial to thoroughly assess their robustness against adversarial inputs. In this work, we introduce Static Deceptor (StaDec) and Dynamic Deceptor (DyDec), two innovative attack frameworks designed to systematically generate dynamic and adaptive adversarial examples by leveraging the understanding of the LLMs. We produce subtle and natural-looking adversarial inputs that preserve semantic similarity to the original text while effectively deceiving the target LLM. By utilizing an automated, LLM-driven pipeline, we eliminate the dependence on external heuristics. Our attacks evolve with the advancements in LLMs and demonstrate strong transferability across models unknown to the attacker. Overall, this work provides a systematic approach for the self-assessment of an LLM's robustness. We release our code and data at https://github.com/Shukti042/AdversarialExample.
Key Contributions
- Static Deceptor (StaDec) and Dynamic Deceptor (DyDec): two fully automated, LLM-driven attack pipelines that generate adaptive adversarial text examples without external heuristics or gradient access
- Adversarial examples that preserve semantic similarity to original text while consistently deceiving state-of-the-art LLM classifiers (GPT-4o, Llama-3-70B) across four sensitive tasks
- Demonstrated strong cross-model transferability and evaluation of three existing defenses against the proposed attacks