RAGRank: Using PageRank to Counter Poisoning in CTI LLM Pipelines

Retrieval-Augmented Generation (RAG) has emerged as the dominant architectural pattern to operationalize Large Language Model (LLM) usage in Cyber Threat Intelligence (CTI) systems. However, this design is susceptible to poisoning attacks, and previously proposed defenses can fail for CTI contexts as cyber threat information is often completely new for emerging attacks, and sophisticated threat actors can mimic legitimate formats, terminology, and stylistic conventions. To address this issue, we propose that the robustness of modern RAG defenses can be accelerated by applying source credibility algorithms on corpora, using PageRank as an example. In our experiments, we demonstrate quantitatively that our algorithm applies a lower authority score to malicious documents while promoting trusted content, using the standardized MS MARCO dataset. We also demonstrate proof-of-concept performance of our algorithm on CTI documents and feeds.

Key Contributions

• RAGRank: a PageRank-derived authority score that suppresses malicious documents and promotes trusted content in RAG corpora
• Citation network construction using explicit citations, LLM-inferred citations, and claim-level entailment, augmented with time decay and author credibility
• Quantitative evaluation on MS MARCO showing lower authority scores for injected malicious documents, plus proof-of-concept on CTI feeds

🛡️ Threat Analysis

Details

Similar Papers