Filtered-ViT: A Robust Defense Against Multiple Adversarial Patch Attacks

Deep learning vision systems are increasingly deployed in safety-critical domains such as healthcare, yet they remain vulnerable to small adversarial patches that can trigger misclassifications. Most existing defenses assume a single patch and fail when multiple localized disruptions occur, the type of scenario adversaries and real-world artifacts often exploit. We propose Filtered-ViT, a new vision transformer architecture that integrates SMART Vector Median Filtering (SMART-VMF), a spatially adaptive, multi-scale, robustness-aware mechanism that enables selective suppression of corrupted regions while preserving semantic detail. On ImageNet with LaVAN multi-patch attacks, Filtered-ViT achieves 79.8% clean accuracy and 46.3% robust accuracy under four simultaneous 1\% patches, outperforming existing defenses. Beyond synthetic benchmarks, a real-world case study on radiographic medical imagery shows that Filtered-ViT mitigates natural artifacts such as occlusions and scanner noise without degrading diagnostic content. This establishes Filtered-ViT as the first transformer to demonstrate unified robustness against both adversarial and naturally occurring patch-like disruptions, charting a path toward reliable vision systems in truly high-stakes environments.

Key Contributions

• Filtered-ViT: a Vision Transformer architecture integrating SMART Vector Median Filtering (SMART-VMF) to selectively suppress adversarial patch regions while preserving semantic content
• First demonstrated defense against multiple simultaneous adversarial patches (up to four 1% patches under LaVAN attack) in a unified transformer framework
• Real-world case study on radiographic medical imagery showing the defense also mitigates natural artifacts like occlusions and scanner noise

🛡️ Threat Analysis

Details

Similar Papers