Read the Scene, Not the Script: Outcome-Aware Safety for LLMs

Safety-aligned Large Language Models (LLMs) still show two dominant failure modes: they are easily jailbroken, or they over-refuse harmless inputs that contain sensitive surface signals. We trace both to a common cause: current models reason weakly about links between actions and outcomes and over-rely on surface-form signals, lexical or stylistic cues that do not encode consequences. We define this failure mode as Consequence-blindness. To study consequence-blindness, we build a benchmark named CB-Bench covering four risk scenarios that vary whether semantic risk aligns with outcome risk, enabling evaluation under both matched and mismatched conditions which are often ignored by existing safety benchmarks. Mainstream models consistently fail to separate these risks and exhibit consequence-blindness, indicating that consequence-blindness is widespread and systematic. To mitigate consequence-blindness, we introduce CS-Chain-4k, a consequence-reasoning dataset for safety alignment. Models fine-tuned on CS-Chain-4k show clear gains against semantic-camouflage jailbreaks and reduce over-refusal on harmless inputs, while maintaining utility and generalization on other benchmarks. These results clarify the limits of current alignment, establish consequence-aware reasoning as a core alignment goal and provide a more practical and reproducible evaluation path.

Key Contributions

• Defines 'consequence-blindness' — LLMs' systematic over-reliance on surface-form lexical signals rather than causal reasoning about real-world outcome risk, unifying jailbreak vulnerability and over-refusal as a single root cause
• CB-Bench: a 600-sample benchmark spanning four risk scenarios that cross-vary semantic risk and outcome risk, enabling evaluation under matched and mismatched conditions overlooked by existing safety benchmarks
• CS-Chain-4k: a consequence-reasoning fine-tuning dataset that improves LLM resistance to semantic-camouflage jailbreaks and reduces over-refusal while preserving general utility

🛡️ Threat Analysis

Details

Similar Papers