Exploring the Security Threats of Retriever Backdoors in Retrieval-Augmented Code Generation

Retrieval-Augmented Code Generation (RACG) is increasingly adopted to enhance Large Language Models for software development, yet its security implications remain dangerously underexplored. This paper conducts the first systematic exploration of a critical and stealthy threat: backdoor attacks targeting the retriever component, which represents a significant supply-chain vulnerability. It is infeasible to assess this threat realistically, as existing attack methods are either too ineffective to pose a real danger or are easily detected by state-of-the-art defense mechanisms spanning both latent-space analysis and token-level inspection, which achieve consistently high detection rates. To overcome this barrier and enable a realistic analysis, we first developed VenomRACG, a new class of potent and stealthy attack that serves as a vehicle for our investigation. Its design makes poisoned samples statistically indistinguishable from benign code, allowing the attack to consistently maintain low detectability across all evaluated defense mechanisms. Armed with this capability, our exploration reveals a severe vulnerability: by injecting vulnerable code equivalent to only 0.05% of the entire knowledge base size, an attacker can successfully manipulate the backdoored retriever to rank the vulnerable code in its top-5 results in 51.29% of cases. This translates to severe downstream harm, causing models like GPT-4o to generate vulnerable code in over 40% of targeted scenarios, while leaving the system's general performance intact. Our findings establish that retriever backdooring is not a theoretical concern but a practical threat to the software development ecosystem that current defenses are blind to, highlighting the urgent need for robust security measures.

Key Contributions

• VenomRACG: a stealthy backdoor attack on RACG retrievers that crafts poisoned code samples statistically indistinguishable from benign code, evading both latent-space and token-level defense mechanisms
• First systematic threat analysis of retriever backdoors in Retrieval-Augmented Code Generation, framing it as a practical supply-chain vulnerability
• Empirical demonstration that 0.05% knowledge base poisoning causes the backdoored retriever to surface vulnerable code in top-5 results 51.29% of the time, leading GPT-4o to generate vulnerable code in over 40% of targeted scenarios

🛡️ Threat Analysis

Details

Similar Papers