Learning to Attack: A Bandit Approach to Adversarial Context Poisoning

Neural contextual bandits are vulnerable to adversarial attacks, where subtle perturbations to rewards, actions, or contexts induce suboptimal decisions. We introduce AdvBandit, a black-box adaptive attack that formulates context poisoning as a continuous-armed bandit problem, enabling the attacker to jointly learn and exploit the victim's evolving policy. The attacker requires no access to the victim's internal parameters, reward function, or gradient information; instead, it constructs a surrogate model using a maximum-entropy inverse reinforcement learning module from observed context-action pairs and optimizes perturbations against this surrogate using projected gradient descent. An upper confidence bound-aware Gaussian process guides arm selection. An attack-budget control mechanism is also introduced to limit detection risk and overhead. We provide theoretical guarantees, including sublinear attacker regret and lower bounds on victim regret linear in the number of attacks. Experiments on three real-world datasets (Yelp, MovieLens, and Disin) against various victim contextual bandits demonstrate that our attack model achieves higher cumulative victim regret than state-of-the-art baselines.

Key Contributions

• AdvBandit: formulates context poisoning as a continuous-armed bandit problem, using GP-UCB over a 3D attack-parameter space to guide PGD-computed perturbations against a surrogate victim model
• UCB-aware Maximum Entropy IRL module that reconstructs the victim's reward function from observed context-action pairs alone, enabling fully black-box operation
• Theoretical guarantees: sublinear attacker regret bound and a victim regret lower bound linear in the number of attacks, plus an attack-budget control mechanism for detection evasion

🛡️ Threat Analysis

Details

Similar Papers