ML Security PapersMitigating Error Amplification in Fast Adversarial Training
Mengnan Zhao 1, Lihe Zhang 2, Bo Wang 2, Tianhang Zheng 3, Hong Zhong 1, Geyong Min 4
Published on arXiv
2604.24332
Input Manipulation Attack
OWASP ML Top 10 — ML01
Key Finding
Alleviates catastrophic overfitting in Fast Adversarial Training while improving the robustness-accuracy trade-off across multiple perturbation budgets
DDG (Distribution-aware Dynamic Guidance)
Novel technique introduced
Fast Adversarial Training (FAT) has proven effective in enhancing model robustness by encouraging networks to learn perturbation-invariant representations. However, FAT often suffers from catastrophic overfitting (CO), where the model overfits to the training attack and fails to generalize to unseen ones. Moreover, robustness oriented optimization typically leads to notable performance degradation on clean inputs, and such degradation becomes increasingly severe as the perturbation budget grows. In this work, we conduct a comprehensive analysis of how guidance strength affects model performance by modulating perturbation and supervision levels across distinct confidence groups. The findings reveal that low confidence samples are the primary contributors to CO and the robustness accuracy trade off. Building on this insight, we propose a Distribution-aware Dynamic Guidance (DDG) strategy that dynamically adjusts both the perturbation budget and supervision signal. Specifically, DDG scales the perturbation magnitude according to the sample confidence at the ground truth class, thereby guiding samples toward consistent decision boundaries while mitigating the influence of learning spurious correlations. Simultaneously, it dynamically adjusts the supervision signal based on the prediction state of each sample, preventing overemphasis on incorrect signals. To alleviate potential gradient instability arising from dynamic guidance, we further design a weighted regularization constraint. Extensive experiments on standard benchmarks demonstrate that DDG effectively alleviates both CO and the robustness accuracy trade off.
Key Contributions
- Distribution-aware Dynamic Guidance (DDG) strategy that adjusts perturbation budgets based on sample confidence
- Dynamic supervision signal adjustment to prevent overemphasis on incorrect signals during adversarial training
- Weighted regularization constraint to stabilize gradients under dynamic guidance
🛡️ Threat Analysis
Addresses adversarial training as a defense against adversarial examples at inference time — prevents catastrophic overfitting where models fail to generalize to unseen adversarial perturbations.