Understanding and Mitigating Over-refusal for Large Language Models via Safety Representation

Large language models demonstrate powerful capabilities across various natural language processing tasks, yet they also harbor safety vulnerabilities. To enhance LLM safety, various jailbreak defense methods have been proposed to guard against harmful outputs. However, improvements in model safety often come at the cost of severe over-refusal, failing to strike a good balance between safety and usability. In this paper, we first analyze the causes of over-refusal from a representation perspective, revealing that over-refusal samples reside at the boundary between benign and malicious samples. Based on this, we propose MOSR, designed to mitigate over-refusal by intervening the safety representation of LLMs. MOSR incorporates two novel components: (1) Overlap-Aware Loss Weighting, which determines the erasure weight for malicious samples by quantifying their similarity to pseudo-malicious samples in the representation space, and (2) Context-Aware Augmentation, which supplements the necessary context for rejection decisions by adding harmful prefixes before rejection responses. Experiments demonstrate that our method outperforms existing approaches in mitigating over-refusal while largely maintaining safety. Overall, we advocate that future defense methods should strike a better balance between safety and over-refusal.

Key Contributions

• Representation-level analysis showing over-refusal samples occupy a boundary region between benign and malicious sample representations in LLMs
• Overlap-Aware Loss Weighting that quantifies similarity of malicious samples to pseudo-malicious samples in representation space to calibrate erasure weights during alignment
• Context-Aware Augmentation that prepends harmful prefixes before rejection responses to provide the contextual signal needed for accurate refusal decisions

🛡️ Threat Analysis

Details

Similar Papers