AegisAgent: An Autonomous Defense Agent Against Prompt Injection Attacks in LLM-HARs

The integration of Large Language Models (LLMs) into wearable sensing is creating a new class of mobile applications capable of nuanced human activity understanding. However, the reliability of these systems is critically undermined by their vulnerability to prompt injection attacks, where attackers deliberately input deceptive instructions into LLMs. Traditional defenses, based on static filters and rigid rules, are insufficient to address the semantic complexity of these new attacks. We argue that a paradigm shift is needed -- from passive filtering to active protection and autonomous reasoning. We introduce AegisAgent, an autonomous agent system designed to ensure the security of LLM-driven HAR systems. Instead of merely blocking threats, AegisAgent functions as a cognitive guardian. It autonomously perceives potential semantic inconsistencies, reasons about the user's true intent by consulting a dynamic memory of past interactions, and acts by generating and executing a multi-step verification and repair plan. We implement AegisAgent as a lightweight, full-stack prototype and conduct a systematic evaluation on 15 common attacks with five state-of-the-art LLM-based HAR systems on three public datasets. Results show it reduces attack success rate by 30\% on average while incurring only 78.6 ms of latency overhead on a GPU workstation. Our work makes the first step towards building secure and trustworthy LLM-driven HAR systems.

Key Contributions

AegisAgent: an autonomous cognitive guardian that perceives semantic inconsistencies in LLM-HAR pipelines, reasons about user intent via dynamic interaction memory, and executes multi-step verification and repair plans against adversarial prompt injection
Systematic evaluation covering 15 prompt injection attack types across 5 state-of-the-art LLM-based HAR systems on 3 public datasets
30% average reduction in attack success rate with only 78.6 ms latency overhead, demonstrating practical viability for wearable/mobile deployment