DARD: Dice Adversarial Robustness Distillation against Adversarial Attacks

Deep learning models are vulnerable to adversarial examples, posing critical security challenges in real-world applications. While Adversarial Training (AT ) is a widely adopted defense mechanism to enhance robustness, it often incurs a trade-off by degrading performance on unperturbed, natural data. Recent efforts have highlighted that larger models exhibit enhanced robustness over their smaller counterparts. In this paper, we empirically demonstrate that such robustness can be systematically distilled from large teacher models into compact student models. To achieve better performance, we introduce Dice Adversarial Robustness Distillation (DARD), a novel method designed to transfer robustness through a tailored knowledge distillation paradigm. Additionally, we propose Dice Projected Gradient Descent (DPGD), an adversarial example generalization method optimized for effective attack. Our extensive experiments demonstrate that the DARD approach consistently outperforms adversarially trained networks with the same architecture, achieving superior robustness and standard accuracy.

Key Contributions

• DARD: a knowledge distillation framework that transfers adversarial robustness from large adversarially-trained teachers to compact student models via dual supervision from natural and adversarial soft labels
• DPGD: adaptation of Dice Projected Gradient Descent from semantic segmentation to image classification with dynamic step-size tuning and channel-wise gradient masking
• Empirical demonstration that DARD-trained lightweight models (ResNet-18) achieve superior robust accuracy on CIFAR-10 and CIFAR-100 over adversarially trained baselines of equivalent architecture

🛡️ Threat Analysis

Details

Similar Papers