ML Security PapersOmniPatch: A Universal Adversarial Patch for ViT-CNN Cross-Architecture Transfer in Semantic Segmentation
Aarush Aggarwal , Akshat Tomar , Amritanshu Tiwari , Sargam Goyal
Published on arXiv
2603.20777
Input Manipulation Attack
OWASP ML Top 10 — ML01
Key Finding
Achieves cross-architecture adversarial patch transfer from ViT to CNN segmentation models using black-box ensemble training
OmniPatch
Novel technique introduced
Robust semantic segmentation is crucial for safe autonomous driving, yet deployed models remain vulnerable to black-box adversarial attacks when target weights are unknown. Most existing approaches either craft image-wide perturbations or optimize patches for a single architecture, which limits their practicality and transferability. We introduce OmniPatch, a training framework for learning a universal adversarial patch that generalizes across images and both ViT and CNN architectures without requiring access to target model parameters.
Key Contributions
- Universal adversarial patch that transfers across both ViT and CNN architectures without target model access
- Sensitive region placement strategy exploiting ViT uncertainty to bias patch positioning
- Two-stage training with gradient alignment for cross-architecture transferability
🛡️ Threat Analysis
Primary contribution is a patch-based adversarial attack that causes misclassification in semantic segmentation models at inference time through spatial perturbations.