ML Security PapersAdaptPrompt: Parameter-Efficient Adaptation of VLMs for Generalizable Deepfake Detection
Yichen Jiang 1, Mohammed Talha Alam 2, Sohail Ahmed Khan 3, Duc-Tien Dang-Nguyen 3, Fakhri Karray 1
Published on arXiv
2512.17730
Output Integrity Attack
OWASP ML Top 10 — ML09
Key Finding
Achieves state-of-the-art across 25 test sets with strong cross-domain generalization, including few-shot performance using as few as 320 training images.
AdaptPrompt
Novel technique introduced
Recent advances in image generation have led to the widespread availability of highly realistic synthetic media, increasing the difficulty of reliable deepfake detection. A key challenge is generalization, as detectors trained on a narrow class of generators often fail when confronted with unseen models. In this work, we address the pressing need for generalizable detection by leveraging large vision-language models, specifically CLIP, to identify synthetic content across diverse generative techniques. First, we introduce Diff-Gen, a large-scale benchmark dataset comprising 100k diffusion-generated fakes that capture broad spectral artifacts unlike traditional GAN datasets. Models trained on Diff-Gen demonstrate stronger cross-domain generalization, particularly on previously unseen image generators. Second, we propose AdaptPrompt, a parameter-efficient transfer learning framework that jointly learns task-specific textual prompts and visual adapters while keeping the CLIP backbone frozen. We further show via layer ablation that pruning the final transformer block of the vision encoder enhances the retention of high-frequency generative artifacts, significantly boosting detection accuracy. Our evaluation spans 25 challenging test sets, covering synthetic content generated by GANs, diffusion models, and commercial tools, establishing a new state-of-the-art in both standard and cross-domain scenarios. We further demonstrate the framework's versatility through few-shot generalization (using as few as 320 images) and source attribution, enabling the precise identification of generator architectures in closed-set settings.
Key Contributions
- AdaptPrompt: a parameter-efficient CLIP adaptation framework that jointly learns textual prompts and visual adapters while keeping the backbone frozen for generalizable deepfake detection
- Diff-Gen: a 100k-image benchmark dataset of diffusion-generated fakes capturing broad spectral artifacts underrepresented in GAN-only training sets
- Layer ablation finding that pruning the final vision encoder transformer block improves retention of high-frequency generative artifacts, boosting cross-domain detection accuracy
🛡️ Threat Analysis
Directly addresses AI-generated content detection (deepfake detection) — the paper's primary contribution is a novel detection architecture for identifying synthetic images produced by GANs, diffusion models, and commercial tools, which falls squarely under output integrity and content authenticity.