ML Security PapersA Unified Spatial Alignment Framework for Highly Transferable Transformation-Based Attacks on Spatially Structured Tasks
Published on arXiv
2603.25230
Input Manipulation Attack
OWASP ML Top 10 — ML01
Key Finding
Degrades average mIoU on Cityscapes from 24.50 to 11.34, on Kvasir-SEG from 49.91 to 31.80, and reduces COCO mAP from 17.89 to 5.25 in non-targeted attacks
Spatial Alignment Framework (SAF)
Novel technique introduced
Transformation-based adversarial attacks (TAAs) demonstrate strong transferability when deceiving classification models. However, existing TAAs often perform unsatisfactorily or even fail when applied to structured tasks such as semantic segmentation and object detection. Encouragingly, recent studies that categorize transformations into non-spatial and spatial transformations inspire us to address this challenge. We find that for non-structured tasks, labels are spatially non-structured, and thus TAAs are not required to adjust labels when applying spatial transformations. In contrast, for structured tasks, labels are spatially structured, and failing to transform labels synchronously with inputs can cause spatial misalignment and yield erroneous gradients. To address these issues, we propose a novel unified Spatial Alignment Framework (SAF) for highly transferable TAAs on spatially structured tasks, where the TAAs spatially transform labels synchronously with the input using the proposed Spatial Alignment (SA) algorithm. Extensive experiments demonstrate the crucial role of our SAF for TAAs on structured tasks. Specifically, in non-targeted attacks, our SAF degrades the average mIoU on Cityscapes from 24.50 to 11.34, and on Kvasir-SEG from 49.91 to 31.80, while reducing the average mAP of COCO from 17.89 to 5.25.
Key Contributions
- Identifies spatial misalignment between inputs and labels as the root cause of TAA failures on structured tasks
- Proposes Spatial Alignment Framework (SAF) that synchronously transforms labels with inputs during adversarial optimization
- Achieves highly transferable attacks on semantic segmentation (Cityscapes, Kvasir-SEG) and object detection (COCO) with significant performance degradation
🛡️ Threat Analysis
Core contribution is adversarial attacks at inference time using spatial transformations (rotation, scaling, translation) to cause misclassification in structured prediction tasks. The Spatial Alignment Framework (SAF) is an attack methodology that applies transformation-based perturbations to deceive segmentation and detection models by synchronously transforming inputs and labels to generate effective adversarial examples. Achieves significant performance degradation: mIoU drops from 24.50 to 11.34 on Cityscapes, and mAP drops from 17.89 to 5.25 on COCO.