How Does the Thinking Step Influence Model Safety? An Entropy-based Safety Reminder for LRMs

Large Reasoning Models (LRMs) achieve remarkable success through explicit thinking steps, yet the thinking steps introduce a novel risk by potentially amplifying unsafe behaviors. Despite this vulnerability, conventional defense mechanisms remain ineffective as they overlook the unique reasoning dynamics of LRMs. In this work, we find that the emergence of safe-reminding phrases within thinking steps plays a pivotal role in ensuring LRM safety. Motivated by this finding, we propose SafeRemind, a decoding-time defense method that dynamically injects safe-reminding phrases into thinking steps. By leveraging entropy triggers to intervene at decision-locking points, SafeRemind redirects potentially harmful trajectories toward safer outcomes without requiring any parameter updates. Extensive evaluations across five LRMs and six benchmarks demonstrate that SafeRemind substantially enhances safety, achieving improvements of up to 45.5%p while preserving core reasoning utility.

Key Contributions

Identifies that safe-reminding phrases within thinking steps are pivotal for LRM safety, and that entropy patterns predict 'decision-locking' points where unsafe trajectories solidify
Proposes SafeRemind, a training-free decoding-time defense that dynamically injects safe-reminding phrases at entropy-triggered intervention points during LRM thinking steps
Demonstrates up to 45.5%p safety improvement across five LRMs and six benchmarks without degrading reasoning utility or requiring parameter updates