GradingAttack: Attacking Large Language Models Towards Short Answer Grading Ability

Large language models (LLMs) have demonstrated remarkable potential for automatic short answer grading (ASAG), significantly boosting student assessment efficiency and scalability in educational scenarios. However, their vulnerability to adversarial manipulation raises critical concerns about automatic grading fairness and reliability. In this paper, we introduce GradingAttack, a fine-grained adversarial attack framework that systematically evaluates the vulnerability of LLM based ASAG models. Specifically, we align general-purpose attack methods with the specific objectives of ASAG by designing token-level and prompt-level strategies that manipulate grading outcomes while maintaining high camouflage. Furthermore, to quantify attack camouflage, we propose a novel evaluation metric that balances attack success and camouflage. Experiments on multiple datasets demonstrate that both attack strategies effectively mislead grading models, with prompt-level attacks achieving higher success rates and token-level attacks exhibiting superior camouflage capability. Our findings underscore the need for robust defenses to ensure fairness and reliability in ASAG. Our code and datasets are available at https://anonymous.4open.science/r/GradingAttack.

Key Contributions

• GradingAttack framework with two complementary attack strategies — gradient-based token-level (white-box) and natural language prompt-level (black-box) — adapted to the ASAG grading context
• Novel camouflage-aware evaluation metric that jointly quantifies attack success rate and degree of obfuscation
• Empirical finding that prompt-level attacks achieve higher success rates while token-level attacks provide superior camouflage on math/science grading datasets

🛡️ Threat Analysis

Details

Similar Papers