ML Security PapersD-REX: A Benchmark for Detecting Deceptive Reasoning in Large Language Models
Satyapriya Krishna 1, Andy Zou 2,3,4, Rahul Gupta 3,4, Eliot Krzysztof Jones 4, Nick Winter 4, Dan Hendrycks 3,4, J. Zico Kolter 1, Matt Fredrikson 2, Spyros Matsoukas 1
Published on arXiv
2509.17938
Prompt Injection
OWASP LLM Top 10 — LLM01
Key Finding
Existing LLMs and safety mechanisms fail to reliably detect deceptive alignment cases in D-REX, where benign outputs mask malicious chain-of-thought induced by adversarial system prompts.
D-REX (Deceptive Reasoning Exposure Suite)
Novel technique introduced
The safety and alignment of Large Language Models (LLMs) are critical for their responsible deployment. Current evaluation methods predominantly focus on identifying and preventing overtly harmful outputs. However, they often fail to address a more insidious failure mode: models that produce benign-appearing outputs while operating on malicious or deceptive internal reasoning. This vulnerability, often triggered by sophisticated system prompt injections, allows models to bypass conventional safety filters, posing a significant, underexplored risk. To address this gap, we introduce the Deceptive Reasoning Exposure Suite (D-REX), a novel dataset designed to evaluate the discrepancy between a model's internal reasoning process and its final output. D-REX was constructed through a competitive red-teaming exercise where participants crafted adversarial system prompts to induce such deceptive behaviors. Each sample in D-REX contains the adversarial system prompt, an end-user's test query, the model's seemingly innocuous response, and, crucially, the model's internal chain-of-thought, which reveals the underlying malicious intent. Our benchmark facilitates a new, essential evaluation task: the detection of deceptive alignment. We demonstrate that D-REX presents a significant challenge for existing models and safety mechanisms, highlighting the urgent need for new techniques that scrutinize the internal processes of LLMs, not just their final outputs.
Key Contributions
- D-REX dataset constructed via red-teaming, containing adversarial system prompts, user queries, benign-appearing model responses, and chain-of-thought revealing malicious intent
- Formalizes 'deceptive alignment detection' as a new evaluation task — detecting discrepancy between internal reasoning and final output
- Demonstrates that existing safety mechanisms fail to detect this class of deceptive behavior, motivating inspection of internal model reasoning