ML Security PapersCHAI: Command Hijacking against embodied AI
Luis Burbano 1, Diego Ortiz 1, Qi Sun 2, Siwei Yang 1, Haoqin Tu 1, Cihang Xie 2, Yinzhi Cao 1, Alvaro A Cardenas 1
Published on arXiv
2510.00181
Input Manipulation Attack
OWASP ML Top 10 — ML01
Prompt Injection
OWASP LLM Top 10 — LLM01
Key Finding
CHAI successfully hijacks LVLM-based control commands across four embodied AI scenarios — including a real robotic vehicle — outperforming prior physical-world adversarial attacks by exploiting multimodal semantic reasoning.
CHAI (Command Hijacking against embodied AI)
Novel technique introduced
Embodied Artificial Intelligence (AI) promises to handle edge cases in robotic vehicle systems where data is scarce by using common-sense reasoning grounded in perception and action to generalize beyond training distributions and adapt to novel real-world situations. These capabilities, however, also create new security risks. In this paper, we introduce CHAI (Command Hijacking against embodied AI), a physical environment indirect prompt injection attack that exploits the multimodal language interpretation abilities of AI models. CHAI embeds deceptive natural language instructions, such as misleading signs, in visual input, systematically searches the token space, builds a dictionary of prompts, and guides an attacker model to generate Visual Attack Prompts. We evaluate CHAI on four LVLM agents: drone emergency landing, autonomous driving, aerial object tracking, and on a real robotic vehicle. Our experiments show that CHAI consistently outperforms state-of-the-art attacks. By exploiting the semantic and multimodal reasoning strengths of next-generation embodied AI systems, CHAI underscores the urgent need for defenses that extend beyond traditional adversarial robustness.
Key Contributions
- CHAI attack framework that systematically searches the token space and builds a dictionary of adversarial prompts to generate Visual Attack Prompts for physical-world indirect prompt injection against LVLM agents
- Demonstration that deceptive natural language signs in the physical environment can reliably hijack text-based control commands in embodied AI systems (drones, autonomous vehicles, robotic platforms)
- Empirical evaluation across four LVLM agent scenarios — including a real robotic vehicle — showing CHAI consistently outperforms state-of-the-art adversarial attacks
🛡️ Threat Analysis
CHAI generates adversarial Visual Attack Prompts — optimized via token-space search — embedded in physical visual scenes to manipulate VLM outputs, constituting adversarial visual input manipulation against multimodal models at inference time.