Exposing Vulnerabilities in RL: A Novel Stealthy Backdoor Attack through Reward Poisoning

Reinforcement learning (RL) has achieved remarkable success across diverse domains, enabling autonomous systems to learn and adapt to dynamic environments by optimizing a reward function. However, this reliance on reward signals creates a significant security vulnerability. In this paper, we study a stealthy backdoor attack that manipulates an agent's policy by poisoning its reward signals. The effectiveness of this attack highlights a critical threat to the integrity of deployed RL systems and calls for urgent defenses against training-time manipulation. We evaluate the attack across classic control and MuJoCo environments. The backdoored agent remains highly stealthy in Hopper and Walker2D, with minimal performance drops of only 2.18 % and 4.59 % under non-triggered scenarios, while achieving strong attack efficacy with up to 82.31% and 71.27% declines under trigger conditions.

Key Contributions

• Novel reward poisoning algorithm formulated as a penalty-based bi-level optimization that minimizes detectable deviations from clean reward data while implanting a backdoor policy.
• Black-box attack requiring no access to the agent's learning algorithm or environment dynamics, making it practically deployable.
• Empirical validation on MuJoCo locomotion tasks (Hopper, Walker2D) demonstrating strong stealthiness (≤4.59% normal performance drop) and high attack efficacy (up to 82.31% decline under triggered conditions).

🛡️ Threat Analysis

Details

Similar Papers