benchmark arXiv Mar 9, 2026 · 28d ago
David Fernandez, Pedram MohajerAnsari, Amir Salarpour et al. · Clemson University
Benchmarks physical adversarial patch attacks across three VLM autonomous driving architectures using black-box NES and semantic homogenization for fair comparison
Input Manipulation Attack Prompt Injection visionmultimodalnlp
Vision-language models are emerging for autonomous driving, yet their robustness to physical adversarial attacks remains unexplored. This paper presents a systematic framework for comparative adversarial evaluation across three VLM architectures: Dolphins, OmniDrive (Omni-L), and LeapVAD. Using black-box optimization with semantic homogenization for fair comparison, we evaluate physically realizable patch attacks in CARLA simulation. Results reveal severe vulnerabilities across all architectures, sustained multi-frame failures, and critical object detection degradation. Our analysis exposes distinct architectural vulnerability patterns, demonstrating that current VLM designs inadequately address adversarial threats in safety-critical autonomous driving applications.
vlm llm transformer Clemson University
ML Security Papers