Soft Instruction De-escalation Defense

Large Language Models (LLMs) are increasingly deployed in agentic systems that interact with an external environment; this makes them susceptible to prompt injections when dealing with untrusted data. To overcome this limitation, we propose SIC (Soft Instruction Control)-a simple yet effective iterative prompt sanitization loop designed for tool-augmented LLM agents. Our method repeatedly inspects incoming data for instructions that could compromise agent behavior. If such content is found, the malicious content is rewritten, masked, or removed, and the result is re-evaluated. The process continues until the input is clean or a maximum iteration limit is reached; if imperative instruction-like content remains, the agent halts to ensure security. By allowing multiple passes, our approach acknowledges that individual rewrites may fail but enables the system to catch and correct missed injections in later steps. Although immediately useful, worst-case analysis shows that SIC is not infallible; strong adversary can still get a 15% ASR by embedding non-imperative workflows. This nonetheless raises the bar.

Key Contributions

• SIC (Soft Instruction Control): an iterative prompt sanitization loop that repeatedly inspects and rewrites/masks/removes injected instructions from external data before agent processing
• Multi-pass design that compensates for individual rewrite failures by re-evaluating until clean or halting if malicious content persists
• Worst-case adversarial analysis showing that even strong adversaries using non-imperative workflow embedding are limited to 15% ASR

🛡️ Threat Analysis

Details

Similar Papers