Divided We Fall: Defending Against Adversarial Attacks via Soft-Gated Fractional Mixture-of-Experts with Randomized Adversarial Training

Machine learning is a powerful tool enabling full automation of a huge number of tasks without explicit programming. Despite recent progress of machine learning in different domains, these models have shown vulnerabilities when they are exposed to adversarial threats. Adversarial threats aim to hinder the machine learning models from satisfying their objectives. They can create adversarial perturbations, which are imperceptible to humans' eyes but have the ability to cause misclassification during inference. In this paper, we propose a defense system, which devises an adversarial training module within mixture-of-experts architecture to enhance its robustness against white-box evasion attacks. In our proposed defense system, we use nine pre-trained classifiers (experts) with ResNet-18 as their backbone. During end-to-end training, the parameters of all experts and the gating mechanism are jointly updated allowing further optimization of the experts. Our proposed defense system outperforms state-of-the-art MoE-based defenses under strong white-box FGSM and PGD evaluation on CIFAR-10 and SVHN.

Key Contributions

• Soft-gated Fractional Mixture-of-Experts (MoE) architecture combining nine ResNet-18 experts trained on benign, FGSM, and PGD regimes for adversarial robustness
• Joint end-to-end training of all expert parameters and the gating mechanism without freezing pretrained weights, improving both clean accuracy and robustness
• Outperforms state-of-the-art MoE-based defenses under strong white-box FGSM and multi-step PGD evaluation on CIFAR-10 and SVHN

🛡️ Threat Analysis

Details

Similar Papers