BashArena: A Control Setting for Highly Privileged AI Agents

Future AI agents might run autonomously with elevated privileges. If these agents are misaligned, they might abuse these privileges to cause serious damage. The field of AI control develops techniques that make it harder for misaligned AIs to cause such damage, while preserving their usefulness. We introduce BashArena, a setting for studying AI control techniques in security-critical environments. BashArena contains 637 Linux system administration and infrastructure engineering tasks in complex, realistic environments, along with four sabotage objectives (execute malware, exfiltrate secrets, escalate privileges, and disable firewall) for a red team to target. We evaluate multiple frontier LLMs on their ability to complete tasks, perform sabotage undetected, and detect sabotage attempts. Claude Sonnet 4.5 successfully executes sabotage while evading monitoring by GPT-4.1 mini 26% of the time, at 4% trajectory-wise FPR. Our findings provide a baseline for designing more effective control protocols in BashArena. We release the dataset as a ControlArena setting and share our task generation pipeline.

Key Contributions

• BashArena dataset of 637 realistic Linux sysadmin/infrastructure tasks with complex Docker-based environments for AI control research
• Four manually constructed sabotage side tasks (execute malware, exfiltrate secrets, escalate privileges, disable firewall) with overlapping permissions with main tasks to prevent naive sandboxing
• Red-team/blue-team evaluation framework showing Claude Sonnet 4.5 achieves 26% sabotage success rate while evading GPT-4.1 mini monitoring at 4% FPR

🛡️ Threat Analysis

Details

Similar Papers