Comparison requires valid measurement: Rethinking attack success rate comparisons in AI red teaming

We argue that conclusions drawn about relative system safety or attack method efficacy via AI red teaming are often not supported by evidence provided by attack success rate (ASR) comparisons. We show, through conceptual, theoretical, and empirical contributions, that many conclusions are founded on apples-to-oranges comparisons or low-validity measurements. Our arguments are grounded in asking a simple question: When can attack success rates be meaningfully compared? To answer this question, we draw on ideas from social science measurement theory and inferential statistics, which, taken together, provide a conceptual grounding for understanding when numerical values obtained through the quantification of system attributes can be meaningfully compared. Through this lens, we articulate conditions under which ASRs can and cannot be meaningfully compared. Using jailbreaking as a running example, we provide examples and extensive discussion of apples-to-oranges ASR comparisons and measurement validity challenges.

Key Contributions

• Identifies conditions under which attack success rate (ASR) comparisons are and are not meaningful in AI red teaming, grounded in social science measurement theory
• Demonstrates that many existing jailbreaking ASR comparisons constitute apples-to-oranges comparisons or low-validity measurements through conceptual, theoretical, and empirical analysis
• Provides a principled framework from inferential statistics and measurement theory for understanding when quantified LLM safety attributes can be legitimately compared

🛡️ Threat Analysis

Details

Similar Papers