AgentSentinel: An End-to-End and Real-Time Security Defense Framework for Computer-Use Agents

Large Language Models (LLMs) have been increasingly integrated into computer-use agents, which can autonomously operate tools on a user's computer to accomplish complex tasks. However, due to the inherently unstable and unpredictable nature of LLM outputs, they may issue unintended tool commands or incorrect inputs, leading to potentially harmful operations. Unlike traditional security risks stemming from insecure user prompts, tool execution results from LLM-driven decisions introduce new and unique security challenges. These vulnerabilities span across all components of a computer-use agent. To mitigate these risks, we propose AgentSentinel, an end-to-end, real-time defense framework designed to mitigate potential security threats on a user's computer. AgentSentinel intercepts all sensitive operations within agent-related services and halts execution until a comprehensive security audit is completed. Our security auditing mechanism introduces a novel inspection process that correlates the current task context with system traces generated during task execution. To thoroughly evaluate AgentSentinel, we present BadComputerUse, a benchmark consisting of 60 diverse attack scenarios across six attack categories. The benchmark demonstrates a 87% average attack success rate on four state-of-the-art LLMs. Our evaluation shows that AgentSentinel achieves an average defense success rate of 79.6%, significantly outperforming all baseline defenses.

Key Contributions

• AgentSentinel: an end-to-end, real-time defense framework that intercepts all sensitive agent operations and halts execution pending a security audit correlating task context with system execution traces
• Novel inspection mechanism that ties system-level traces back to the originating task context to detect anomalous or harmful agent behavior
• BadComputerUse: a benchmark of 60 diverse attack scenarios across 6 attack categories, establishing an 87% average attack success rate baseline on 4 state-of-the-art LLMs

🛡️ Threat Analysis

Details

Similar Papers