ML Security PapersRIPRAG: Hack a Black-box Retrieval-Augmented Generation Question-Answering System with Reinforcement Learning
Meng Xi , Sihan Lv , Yechen Jin , Guanjie Cheng , Naibo Wang , Ying Li , Jianwei Yin
Published on arXiv
2510.10008
Data Poisoning Attack
OWASP ML Top 10 — ML02
Prompt Injection
OWASP LLM Top 10 — LLM01
Key Finding
RIPRAG achieves an attack success rate improvement of up to 0.72 over baseline black-box methods, including against complex RAG systems with advanced defenses where white-box methods are inapplicable.
RIPRAG
Novel technique introduced
Retrieval-Augmented Generation (RAG) systems based on Large Language Models (LLMs) have become a core technology for tasks such as question-answering (QA) and content generation. RAG poisoning is an attack method to induce LLMs to generate the attacker's expected text by injecting poisoned documents into the database of RAG systems. Existing research can be broadly divided into two classes: white-box methods and black-box methods. White-box methods utilize gradient information to optimize poisoned documents, and black-box methods use a pre-trained LLM to generate them. However, existing white-box methods require knowledge of the RAG system's internal composition and implementation details, whereas black-box methods are unable to utilize interactive information. In this work, we propose the RIPRAG attack framework, an end-to-end attack pipeline that treats the target RAG system as a black box and leverages our proposed Reinforcement Learning from Black-box Feedback (RLBF) method to optimize the generation model for poisoned documents. We designed two kinds of rewards: similarity reward and attack reward. Experimental results demonstrate that this method can effectively execute poisoning attacks against most complex RAG systems, achieving an attack success rate (ASR) improvement of up to 0.72 compared to baseline methods. This highlights prevalent deficiencies in current defensive methods and provides critical insights for LLM security research.
Key Contributions
- RIPRAG: first RL-based end-to-end black-box attack framework for poisoning RAG systems, using only input-output queries to the target system.
- RLBF (Reinforcement Learning from Black-box Feedback): a novel RL paradigm with similarity and attack rewards to iteratively refine poisoned document generation without access to system internals.
- BRPO (Batch Relative Policy Optimization): a new policy optimization algorithm for stable and efficient adversarial text generation, with the first rigorous benchmark of attacks against RAG systems equipped with advanced targeted defenses.
🛡️ Threat Analysis
The core attack mechanism is injecting malicious documents into the RAG system's retrieval database — a form of data poisoning that corrupts the knowledge store the model relies on to generate responses.