Adversarial Attacks Leverage Interference Between Features in Superposition

Fundamental questions remain about when and why adversarial examples arise in neural networks, with competing views characterising them either as artifacts of the irregularities in the decision landscape or as products of sensitivity to non-robust input features. In this paper, we instead argue that adversarial vulnerability can stem from efficient information encoding in neural networks. Specifically, we show how superposition - where networks represent more features than they have dimensions - creates arrangements of latent representations that adversaries can exploit. We demonstrate that adversarial perturbations leverage interference between superposed features, making attack patterns predictable from feature arrangements. Our framework provides a mechanistic explanation for two known phenomena: adversarial attack transferability between models with similar training regimes and class-specific vulnerability patterns. In synthetic settings with precisely controlled superposition, we establish that superposition suffices to create adversarial vulnerability. We then demonstrate that these findings persist in a ViT trained on CIFAR-10. These findings reveal adversarial vulnerability can be a byproduct of networks' representational compression, rather than flaws in the learning process or non-robust inputs.

Key Contributions

• Mechanistic theory linking neural superposition (more features than dimensions) to adversarial vulnerability via inter-feature interference in latent space
• Analytical explanation of two known adversarial phenomena — cross-model transferability and class-specific vulnerability — through the lens of feature arrangement geometry
• Empirical validation in controlled synthetic superposition settings and a ViT trained on CIFAR-10, showing superposition alone is sufficient to induce adversarial vulnerability

🛡️ Threat Analysis

Details

Similar Papers