On damage of interpolation to adversarial robustness in regression

Deep neural networks (DNNs) typically involve a large number of parameters and are trained to achieve zero or near-zero training error. Despite such interpolation, they often exhibit strong generalization performance on unseen data, a phenomenon that has motivated extensive theoretical investigations. Comforting results show that interpolation indeed may not affect the minimax rate of convergence under the squared error loss. In the mean time, DNNs are well known to be highly vulnerable to adversarial perturbations in future inputs. A natural question then arises: Can interpolation also escape from suboptimal performance under a future $X$-attack? In this paper, we investigate the adversarial robustness of interpolating estimators in a framework of nonparametric regression. A finding is that interpolating estimators must be suboptimal even under a subtle future $X$-attack, and achieving perfect fitting can substantially damage their robustness. An interesting phenomenon in the high interpolation regime, which we term the curse of simple size, is also revealed and discussed. Numerical experiments support our theoretical findings.

Key Contributions

• Establishes minimax adversarial L2-risk theory for the class of interpolating estimators in nonparametric regression, proving they are provably suboptimal under adversarial X-attacks
• Reveals a 'curse of sample size' phenomenon: in high interpolation regimes, more training data deteriorates adversarial robustness of interpolators
• Shows that any interpolating method (including over-parameterized DNNs achieving near-zero training error) cannot attain optimal adversarial robustness, identifying interpolation as a structural cause of adversarial vulnerability

🛡️ Threat Analysis

Details

Similar Papers