ML Security PapersRAGShield: Provenance-Verified Defense-in-Depth Against Knowledge Base Poisoning in Government Retrieval-Augmented Generation Systems
Published on arXiv
2604.00387
Data Poisoning Attack
OWASP ML Top 10 — ML02
Training Data Poisoning
OWASP LLM Top 10 — LLM03
Key Finding
Achieves 0.0% attack success rate (95% CI: [0.0%, 1.9%]) against five adversary tiers including adaptive attacks, with 0.0% false positive rate
RAGShield
Novel technique introduced
RAG systems deployed across federal agencies for citizen-facing services are vulnerable to knowledge base poisoning attacks, where adversaries inject malicious documents to manipulate outputs. Recent work demonstrates that as few as 10 adversarial passages can achieve 98.2% retrieval success rates. We observe that RAG knowledge base poisoning is structurally analogous to software supply chain attacks, and propose RAGShield, a five-layer defense-in-depth framework applying supply chain provenance verification to the RAG knowledge pipeline. RAGShield introduces: (1) C2PA-inspired cryptographic document attestation blocking unsigned and forged documents at ingestion; (2) trust-weighted retrieval prioritizing provenance-verified sources; (3) a formal taint lattice with cross-source contradiction detection catching insider threats even when provenance is valid; (4) provenance-aware generation with auditable citations; and (5) NIST SP 800-53 compliance mapping across 15 control families. Evaluation on a 500-passage Natural Questions corpus with 63 attack documents and 200 queries against five adversary tiers achieves 0.0% attack success rate including adaptive attacks (95% CI: [0.0%, 1.9%]) with 0.0% false positive rate. We honestly report that insider in-place replacement attacks achieve 17.5% ASR, identifying the fundamental limit of ingestion-time defense. The cross-source contradiction detector catches subtle numerical manipulation attacks that bypass provenance verification entirely.
Key Contributions
- First application of supply chain provenance verification (C2PA-inspired cryptographic attestation) to RAG security
- Five-layer defense-in-depth framework including taint lattice with cross-source contradiction detection for insider threats
- First NIST SP 800-53 compliance mapping for RAG pipeline security across 15 control families
🛡️ Threat Analysis
RAG knowledge base poisoning is data poisoning—adversaries inject malicious documents into the RAG corpus to manipulate retrieval and generation outputs. The paper defends against external injection, insider compromise, and data loader exploitation attacks that corrupt the knowledge base at ingestion/training time.