Optimizing AI Agent Attacks With Synthetic Data

As AI deployments become more complex and high-stakes, it becomes increasingly important to be able to estimate their risk. AI control is one framework for doing so. However, good control evaluations require eliciting strong attack policies. This can be challenging in complex agentic environments where compute constraints leave us data-poor. In this work, we show how to optimize attack policies in SHADE-Arena, a dataset of diverse realistic control environments. We do this by decomposing attack capability into five constituent skills -- suspicion modeling, attack selection, plan synthesis, execution and subtlety -- and optimizing each component individually. To get around the constraint of limited data, we develop a probabilistic model of attack dynamics, optimize our attack hyperparameters using this simulation, and then show that the results transfer to SHADE-Arena. This results in a substantial improvement in attack strength, reducing safety score from a baseline of 0.87 to 0.41 using our scaffold.

Key Contributions

Decomposes attack capability into five modular skills (suspicion modeling, attack selection, plan synthesis, execution, subtlety) enabling targeted optimization of each component independently.
Develops a probabilistic simulation model of attack dynamics fitted to real SHADE-Arena transcripts, enabling hyperparameter optimization and counterfactual reasoning without expensive end-to-end rollouts.
Demonstrates stacking of modular interventions reduces safety score from 0.87 to 0.41 on SHADE-Arena, more than doubling attack strength over the baseline policy.