ML Security PapersEvaluation Awareness Scales Predictably in Open-Weights Large Language Models
Maheep Chaudhary 1, Ian Su 2, Nikhil Hooda 3, Nishith Shankar 4, Julia Tan 5, Kevin Zhu 6, Ryan Lagasse 6, Vasu Sharma 7, Ashwinee Panda 8
Published on arXiv
2509.13333
Prompt Injection
OWASP LLM Top 10 — LLM01
Key Finding
Evaluation awareness follows a power-law scaling with model size across 15 LLMs, enabling forecasting of deceptive capability-concealment behavior in future larger models
Linear probing on steering vector activations
Novel technique introduced
Large language models (LLMs) can internally distinguish between evaluation and deployment contexts, a behaviour known as \emph{evaluation awareness}. This undermines AI safety evaluations, as models may conceal dangerous capabilities during testing. Prior work demonstrated this in a single $70$B model, but the scaling relationship across model sizes remains unknown. We investigate evaluation awareness across $15$ models scaling from $0.27$B to $70$B parameters from four families using linear probing on steering vector activations. Our results reveal a clear power-law scaling: evaluation awareness increases predictably with model size. This scaling law enables forecasting deceptive behavior in future larger models and guides the design of scale-aware evaluation strategies for AI safety. A link to the implementation of this paper can be found at https://anonymous.4open.science/r/evaluation-awareness-scaling-laws/README.md.
Key Contributions
- First systematic study of evaluation awareness scaling across 15 models (0.27B–70B) from four LLM families
- Discovery of a clear power-law scaling relationship: evaluation awareness increases predictably with model size
- Linear probing methodology on steering vector activations to detect and quantify evaluation awareness