ML Security PapersPolicy Compiler for Secure Agentic Systems
Nils Palumbo 1, Sarthak Choudhary 1, Jihye Choi 1, Prasad Chalasani 2, Somesh Jha 1
Published on arXiv
2602.16708
Excessive Agency
OWASP LLM Top 10 — LLM08
Prompt Injection
OWASP LLM Top 10 — LLM01
Key Finding
PCAS improves policy compliance from 48% to 93% across frontier models on customer service tasks, with zero policy violations in instrumented runs
PCAS
Novel technique introduced
LLM-based agents are increasingly being deployed in contexts requiring complex authorization policies: customer service protocols, approval workflows, data access restrictions, and regulatory compliance. Embedding these policies in prompts provides no enforcement guarantees. We present PCAS, a Policy Compiler for Agentic Systems that provides deterministic policy enforcement. Enforcing such policies requires tracking information flow across agents, which linear message histories cannot capture. Instead, PCAS models the agentic system state as a dependency graph capturing causal relationships among events such as tool calls, tool results, and messages. Policies are expressed in a Datalog-derived language, as declarative rules that account for transitive information flow and cross-agent provenance. A reference monitor intercepts all actions and blocks violations before execution, providing deterministic enforcement independent of model reasoning. PCAS takes an existing agent implementation and a policy specification, and compiles them into an instrumented system that is policy-compliant by construction, with no security-specific restructuring required. We evaluate PCAS on three case studies: information flow policies for prompt injection defense, approval workflows in a multi-agent pharmacovigilance system, and organizational policies for customer service. On customer service tasks, PCAS improves policy compliance from 48% to 93% across frontier models, with zero policy violations in instrumented runs.
Key Contributions
- PCAS: a policy compiler that instruments existing LLM agent implementations into policy-compliant systems by construction, with no security-specific restructuring required
- Dependency graph formalism capturing causal relationships among tool calls, tool results, and messages to track transitive information flow across agents
- Datalog-derived declarative policy language combined with a reference monitor that deterministically blocks policy violations before execution