ML Security PapersPhysical ID-Transfer Attacks against Multi-Object Tracking via Adversarial Trajectory
Chenyi Wang 1, Yanmao Man 1,2, Raymond Muller 3, Ming Li 1, Z. Berkay Celik 3, Ryan Gerdes 4, Jonathan Petit 5
Published on arXiv
2512.01934
Input Manipulation Attack
OWASP ML Top 10 — ML01
Key Finding
AdvTraj achieves 100% attack success rate in white-box attacks against SORT in CARLA simulation and up to 93% transferability against state-of-the-art MOT algorithms due to common design principles in their association modules
AdvTraj
Novel technique introduced
Multi-Object Tracking (MOT) is a critical task in computer vision, with applications ranging from surveillance systems to autonomous driving. However, threats to MOT algorithms have yet been widely studied. In particular, incorrect association between the tracked objects and their assigned IDs can lead to severe consequences, such as wrong trajectory predictions. Previous attacks against MOT either focused on hijacking the trackers of individual objects, or manipulating the tracker IDs in MOT by attacking the integrated object detection (OD) module in the digital domain, which are model-specific, non-robust, and only able to affect specific samples in offline datasets. In this paper, we present AdvTraj, the first online and physical ID-manipulation attack against tracking-by-detection MOT, in which an attacker uses adversarial trajectories to transfer its ID to a targeted object to confuse the tracking system, without attacking OD. Our simulation results in CARLA show that AdvTraj can fool ID assignments with 100% success rate in various scenarios for white-box attacks against SORT, which also have high attack transferability (up to 93% attack success rate) against state-of-the-art (SOTA) MOT algorithms due to their common design principles. We characterize the patterns of trajectories generated by AdvTraj and propose two universal adversarial maneuvers that can be performed by a human walker/driver in daily scenarios. Our work reveals under-explored weaknesses in the object association phase of SOTA MOT systems, and provides insights into enhancing the robustness of such systems.
Key Contributions
- First online, physical ID-transfer attack against tracking-by-detection MOT that targets the association phase without attacking the integrated object detector
- AdvTraj algorithm that generates adversarial trajectories achieving 100% ID-swap success rate against SORT and up to 93% transferability against SOTA MOT algorithms
- Characterization of universal adversarial maneuvers — simple physical movement patterns performable by a human walker or driver in real-world scenarios
🛡️ Threat Analysis
AdvTraj crafts adversarial physical inputs (movement trajectories) that manipulate MOT system outputs (ID assignments) at inference time — a physical input manipulation attack exploiting the object-association phase of tracking algorithms rather than the object detector.