defense arXiv Jan 11, 2025 · Jan 2025
Wenshu Fan, Minxing Zhang, Hongwei Li et al. · University of Electronic Science and Technology of China · CISPA Helmholtz Center for Information Security +1 more
Introduces adaptive gallery-update attack breaking all AFR defenses, then counters with diverse adversarial perturbations for facial privacy
Input Manipulation Attack vision
The widespread adoption of facial recognition (FR) models raises serious concerns about their potential misuse, motivating the development of anti-facial recognition (AFR) to protect user facial privacy. In this paper, we argue that the static FR strategy, predominantly adopted in prior literature for evaluating AFR efficacy, cannot faithfully characterize the actual capabilities of determined trackers who aim to track a specific target identity. In particular, we introduce DynTracker, a dynamic FR strategy where the model's gallery database is iteratively updated with newly recognized target identity images. Surprisingly, such a simple approach renders all the existing AFR protections ineffective. To mitigate the privacy threats posed by DynTracker, we advocate for explicitly promoting diversity in the AFR-protected images. We hypothesize that the lack of diversity is the primary cause of the failure of existing AFR methods. Specifically, we develop DivTrackee, a novel method for crafting diverse AFR protections that builds upon a text-guided image generation framework and diversity-promoting adversarial losses. Through comprehensive experiments on various image benchmarks and feature extractors, we demonstrate DynTracker's strength in breaking existing AFR methods and the superiority of DivTrackee in preventing user facial images from being identified by dynamic FR strategies. We believe our work can act as an important initial step towards developing more effective AFR methods for protecting user facial privacy against determined trackers.
cnn diffusion University of Electronic Science and Technology of China · CISPA Helmholtz Center for Information Security · The Chinese University of Hong Kong
attack arXiv Mar 12, 2026 · 25d ago
Zikang Ding, Haomiao Yang, Meng Hao et al. · University of Electronic Science and Technology of China · Singapore Management University +2 more
Proposes temporally-delayed backdoor attacks on NLP pre-trained models using common everyday words as stealthy triggers
Model Poisoning nlp
Backdoor attacks against pre-trained models (PTMs) have traditionally operated under an ``immediacy assumption,'' where malicious behavior manifests instantly upon trigger occurrence. This work revisits and challenges this paradigm by introducing \textit{\textbf{Delayed Backdoor Attacks (DBA)}}, a new class of threats in which activation is temporally decoupled from trigger exposure. We propose that this \textbf{temporal dimension} is the key to unlocking a previously infeasible class of attacks: those that use common, everyday words as triggers. To examine the feasibility of this paradigm, we design and implement a proof-of-concept prototype, termed \underline{D}elayed Backdoor Attacks Based on \underline{N}onlinear \underline{D}ecay (DND). DND embeds a lightweight, stateful logic module that postpones activation until a configurable threshold is reached, producing a distinct latency phase followed by a controlled outbreak. We derive a formal model to characterize this latency behavior and propose a dual-metric evaluation framework (ASR and ASR$_{delay}$) to empirically measure the delay effect. Extensive experiments on four (natural language processing)NLP benchmarks validate the core capabilities of DND: it remains dormant for a controllable duration, sustains high clean accuracy ($\ge$94\%), and achieves near-perfect post-activation attack success rates ($\approx$99\%, The average of other methods is below 95\%.). Moreover, DND exhibits resilience against several state-of-the-art defenses. This study provides the first empirical evidence that the temporal dimension constitutes a viable yet unprotected attack surface in PTMs, underscoring the need for next-generation, stateful, and time-aware defense mechanisms.
transformer llm University of Electronic Science and Technology of China · Singapore Management University · Xi’an Polytechnic University +1 more
attack arXiv Aug 6, 2025 · Aug 2025
Kunlan Xiang, Haomiao Yang, Meng Hao et al. · University of Electronic Science and Technology of China · Singapore Management University +3 more
Proposes first backdoor attack on multivariate time series forecasting, extending attackable horizon 60× to 720 timesteps via lag-aware distributed triggers
Model Poisoning Data Poisoning Attack timeseries
Multivariate long-term time series forecasting (MLTSF) models are increasingly deployed in critical domains such as climate, finance, and transportation. Despite their growing importance, the security of MLTSF models against backdoor attacks remains entirely unexplored. To bridge this gap, we propose BadTime, the first effective backdoor attack tailored for MLTSF. BadTime can manipulate hundreds of future predictions toward a target pattern by injecting a subtle trigger. BadTime addresses two key challenges that arise uniquely in MLTSF: (i) the rapid dilution of local triggers over long horizons, and (ii) the extreme sparsity of backdoor signals under stealth constraints. To counter dilution, BadTime leverages inter-variable correlations, temporal lags, and data-driven initialization to design a distributed, lag-aware trigger that ensures effective influence over long-range forecasts. To overcome sparsity, it introduces a hybrid strategy to select valuable poisoned samples and a decoupled backdoor training objective that adaptively adjusts the model's focus on the sparse backdoor signal, ensuring reliable learning at a poisoning rate as low as 1%. Extensive experiments show that BadTime significantly outperforms state-of-the-art (SOTA) backdoor attacks on time series forecasting by extending the attackable horizon from at most 12 timesteps to 720 timesteps (a 60-fold improvement), reducing MAE by over 50% on target variables, and boosting stealthiness by more than 3-fold under anomaly detection.
transformer University of Electronic Science and Technology of China · Singapore Management University · Sichuan University +2 more
attack arXiv Aug 26, 2025 · Aug 2025
Rui Zhang, Zihan Wang, Tianli Yang et al. · University of Electronic Science and Technology of China · City University of Hong Kong +1 more
Adversarial image attack on VLMs that maximizes output length via hidden special tokens, exhausting inference resources stealthily
Input Manipulation Attack Model Denial of Service visionmultimodalnlp
Vision-Language Models (VLMs) are increasingly deployed in real-world applications, but their high inference cost makes them vulnerable to resource consumption attacks. Prior attacks attempt to extend VLM output sequences by optimizing adversarial images, thereby increasing inference costs. However, these extended outputs often introduce irrelevant abnormal content, compromising attack stealthiness. This trade-off between effectiveness and stealthiness poses a major limitation for existing attacks. To address this challenge, we propose \textit{Hidden Tail}, a stealthy resource consumption attack that crafts prompt-agnostic adversarial images, inducing VLMs to generate maximum-length outputs by appending special tokens invisible to users. Our method employs a composite loss function that balances semantic preservation, repetitive special token induction, and suppression of the end-of-sequence (EOS) token, optimized via a dynamic weighting strategy. Extensive experiments show that \textit{Hidden Tail} outperforms existing attacks, increasing output length by up to 19.2$\times$ and reaching the maximum token limit, while preserving attack stealthiness. These results highlight the urgent need to improve the robustness of VLMs against efficiency-oriented adversarial threats. Our code is available at https://github.com/zhangrui4041/Hidden_Tail.
vlm llm transformer University of Electronic Science and Technology of China · City University of Hong Kong · Nanyang Technological University
defense arXiv Aug 2, 2025 · Aug 2025
Zihan Wang, Rui Zhang, Hongwei Li et al. · University of Electronic Science and Technology of China · City University of Hong Kong
Detects LLM backdoors in real-time by monitoring token confidence windows that reveal the 'sequence lock' phenomenon
Model Poisoning nlp
Backdoor attacks pose a significant threat to Large Language Models (LLMs), where adversaries can embed hidden triggers to manipulate LLM's outputs. Most existing defense methods, primarily designed for classification tasks, are ineffective against the autoregressive nature and vast output space of LLMs, thereby suffering from poor performance and high latency. To address these limitations, we investigate the behavioral discrepancies between benign and backdoored LLMs in output space. We identify a critical phenomenon which we term sequence lock: a backdoored model generates the target sequence with abnormally high and consistent confidence compared to benign generation. Building on this insight, we propose ConfGuard, a lightweight and effective detection method that monitors a sliding window of token confidences to identify sequence lock. Extensive experiments demonstrate ConfGuard achieves a near 100\% true positive rate (TPR) and a negligible false positive rate (FPR) in the vast majority of cases. Crucially, the ConfGuard enables real-time detection almost without additional latency, making it a practical backdoor defense for real-world LLM deployments.
llm transformer University of Electronic Science and Technology of China · City University of Hong Kong