ML Security PapersTransferable Direct Prompt Injection via Activation-Guided MCMC Sampling
Minghui Li 1, Hao Zhang 1, Yechao Zhang 2, Wei Wan 3, Shengshan Hu 1, pei Xiaobing 1, Jing Wang 1
Published on arXiv
2509.07617
Prompt Injection
OWASP LLM Top 10 — LLM01
Key Finding
Achieves 49.6% ASR across five mainstream LLMs and 34.6% improvement over human-crafted prompts, with 36.6% ASR on unseen task scenarios
Activation-Guided MCMC DPI
Novel technique introduced
Direct Prompt Injection (DPI) attacks pose a critical security threat to Large Language Models (LLMs) due to their low barrier of execution and high potential damage. To address the impracticality of existing white-box/gray-box methods and the poor transferability of black-box methods, we propose an activations-guided prompt injection attack framework. We first construct an Energy-based Model (EBM) using activations from a surrogate model to evaluate the quality of adversarial prompts. Guided by the trained EBM, we employ the token-level Markov Chain Monte Carlo (MCMC) sampling to adaptively optimize adversarial prompts, thereby enabling gradient-free black-box attacks. Experimental results demonstrate our superior cross-model transferability, achieving 49.6% attack success rate (ASR) across five mainstream LLMs and 34.6% improvement over human-crafted prompts, and maintaining 36.6% ASR on unseen task scenarios. Interpretability analysis reveals a correlation between activations and attack effectiveness, highlighting the critical role of semantic patterns in transferable vulnerability exploitation.
Key Contributions
- First transferable DPI attack guided by surrogate model activations, eliminating the need to query the victim model directly
- Energy-based Model (EBM) constructed from surrogate activations to score and guide adversarial prompt quality
- Token-level MCMC sampling strategy for gradient-free optimization of natural adversarial prompts with high cross-model transferability