ML Security PapersFrom Rookie to Expert: Manipulating LLMs for Automated Vulnerability Exploitation in Enterprise Software
Moustapha Awwalou Diouf 1, Maimouna Tamah Diao 1, Iyiola E. Olatunji 1, Abdoul Kader Kaboré 1, Jordan Samhi 1, Gervais Mendy 2, Samuel Ouya 3, Jacques Klein 1, Tegawendé F. Bissyandé 1
Published on arXiv
2512.22753
Prompt Injection
OWASP LLM Top 10 — LLM01
Key Finding
Achieved 100% exploit generation success rate across five LLMs on Odoo CVEs within 3-4 prompting rounds using the RSA pretexting strategy
RSA (Role-assignment, Scenario-pretexting, Action-solicitation)
Novel technique introduced
LLMs democratize software engineering by enabling non-programmers to create applications, but this same accessibility fundamentally undermines security assumptions that have guided software engineering for decades. We show in this work how publicly available LLMs can be socially engineered to transform novices into capable attackers, challenging the foundational principle that exploitation requires technical expertise. To that end, we propose RSA (Role-assignment, Scenario-pretexting, and Action-solicitation), a pretexting strategy that manipulates LLMs into generating functional exploits despite their safety mechanisms. Testing against Odoo -- a widely used ERP platform, we evaluated five mainstream LLMs (GPT-4o, Gemini, Claude, Microsoft Copilot, and DeepSeek) and achieved a 100% success rate: tested CVE yielded at least one working exploit within 3-4 prompting rounds. While prior work [13] found LLM-assisted attacks difficult and requiring manual effort, we demonstrate that this overhead can be eliminated entirely. Our findings invalidate core software engineering security principles: the distinction between technical and non-technical actors no longer provides valid threat models; technical complexity of vulnerability descriptions offers no protection when LLMs can abstract it away; and traditional security boundaries dissolve when the same tools that build software can be manipulated to break it. This represents a paradigm shift in software engineering -- we must redesign security practices for an era where exploitation requires only the ability to craft prompts, not understand code. Artifacts available at: https://anonymous.4open.science/r/From-Rookie-to-Attacker-D8B3.
Key Contributions
- Proposes RSA (Role-assignment, Scenario-pretexting, Action-solicitation), a structured pretexting strategy to manipulate LLMs into generating functional exploits despite safety guardrails
- Demonstrates 100% jailbreak success rate across five major LLMs (GPT-4o, Gemini, Claude, Copilot, DeepSeek) for generating working CVE exploits against Odoo ERP in 3-4 prompting rounds
- Challenges foundational software security assumptions by showing that technical expertise is no longer a barrier to exploitation when LLMs can abstract away complexity