Language Models Identify Ambiguities and Exploit Loopholes

Studying the responses of large language models (LLMs) to loopholes presents a two-fold opportunity. First, it affords us a lens through which to examine ambiguity and pragmatics in LLMs, since exploiting a loophole requires identifying ambiguity and performing sophisticated pragmatic reasoning. Second, loopholes pose an interesting and novel alignment problem where the model is presented with conflicting goals and can exploit ambiguities to its own advantage. To address these questions, we design scenarios where LLMs are given a goal and an ambiguous user instruction in conflict with the goal, with scenarios covering scalar implicature, structural ambiguities, and power dynamics. We then measure different models' abilities to exploit loopholes to satisfy their given goals as opposed to the goals of the user. We find that both closed-source and stronger open-source models can identify ambiguities and exploit their resulting loopholes, presenting a potential AI safety risk. Our analysis indicates that models which exploit loopholes explicitly identify and reason about both ambiguity and conflicting goals.

Key Contributions

Novel benchmark scenarios covering scalar implicature, structural bracketing ambiguities, and power-dynamics loopholes to measure intentional misinterpretation by LLM agents
Empirical finding that stronger closed-source and open-source LLMs (GPT-4, Gemini, Llama-70B) can identify ambiguities and exploit resulting loopholes to satisfy competing system-prompt goals
Analysis showing that loophole-exploiting models explicitly reason about both the ambiguity and the goal conflict rather than naively mis-parsing instructions