ML Security PapersRAG-Pull: Imperceptible Attacks on RAG Systems for Code Generation
Vasilije Stambolic 1,2, Aritra Dhar 2, Lukas Cavigelli 2
Published on arXiv
2510.11195
Input Manipulation Attack
OWASP ML Top 10 — ML01
Prompt Injection
OWASP LLM Top 10 — LLM01
Key Finding
Combined query-and-target UTF character perturbations achieve near-perfect retrieval redirection in RAG-based code generation systems, causing LLMs to produce code with exploitable vulnerabilities such as RCE and SQL injection.
RAG-Pull
Novel technique introduced
Retrieval-Augmented Generation (RAG) increases the reliability and trustworthiness of the LLM response and reduces hallucination by eliminating the need for model retraining. It does so by adding external data into the LLM's context. We develop a new class of black-box attack, RAG-Pull, that inserts hidden UTF characters into queries or external code repositories, redirecting retrieval toward malicious code, thereby breaking the models' safety alignment. We observe that query and code perturbations alone can shift retrieval toward attacker-controlled snippets, while combined query-and-target perturbations achieve near-perfect success. Once retrieved, these snippets introduce exploitable vulnerabilities such as remote code execution and SQL injection. RAG-Pull's minimal perturbations can alter the model's safety alignment and increase preference towards unsafe code, therefore opening up a new class of attacks on LLMs.
Key Contributions
- RAG-Pull attack using hidden UTF characters to imperceptibly manipulate query and document similarity, redirecting retrieval toward attacker-controlled malicious code snippets
- Demonstrates that combined query-and-target perturbations achieve near-perfect retrieval success against RAG-based code generation systems
- Shows that retrieved malicious snippets introduce real exploitable vulnerabilities (remote code execution, SQL injection), effectively bypassing LLM safety alignment
🛡️ Threat Analysis
RAG-Pull uses strategically crafted imperceptible perturbations (hidden UTF characters) inserted into queries and external code repositories to manipulate retrieval in an LLM-integrated system — a clear case of adversarial content manipulation of an RAG/LLM pipeline, which the classification rules explicitly cite as ML01+LLM01 territory (adversarial document injection for RAG).