ML Security PapersDiffusionFF: A Diffusion-based Framework for Joint Face Forgery Detection and Fine-Grained Artifact Localization
Siran Peng 1,2, Haoyuan Zhang 2,1, Li Gao 3, Tianshuo Zhang 2,1, Xiangyu Zhu 1,2, Bao Li 1,2, Weisong Zhao 4,5, Zhen Lei 1,2,6,7
1 Institute of Automation, Chinese Academy of Sciences
2 University of Chinese Academy of Sciences
3 China Mobile Research Institute
4 Institute of Information Engineering, Chinese Academy of Sciences
5 School of Cyber Security, University of Chinese Academy of Sciences
6 Hong Kong Institute of Science and Innovation, Chinese Academy of Sciences
Published on arXiv
2508.01873
Output Integrity Attack
OWASP ML Top 10 — ML09
Key Finding
DiffusionFF achieves state-of-the-art performance across multiple deepfake detection benchmarks by conditioning a diffusion decoder on multi-scale forgery features to produce detailed artifact localization maps.
DiffusionFF
Novel technique introduced
The rapid evolution of deepfake technologies demands robust and reliable face forgery detection algorithms. While determining whether an image has been manipulated remains essential, the ability to precisely localize forgery clues is also important for enhancing model explainability and building user trust. To address this dual challenge, we introduce DiffusionFF, a diffusion-based framework that simultaneously performs face forgery detection and fine-grained artifact localization. Our key idea is to establish a novel encoder-decoder architecture: a pretrained forgery detector serves as a powerful "artifact encoder", and a denoising diffusion model is repurposed as an "artifact decoder". Conditioned on multi-scale forgery-related features extracted by the encoder, the decoder progressively synthesizes a detailed artifact localization map. We then fuse this fine-grained localization map with high-level semantic features from the forgery detector, leading to substantial improvements in detection capability. Extensive experiments show that DiffusionFF achieves state-of-the-art (SOTA) performance across multiple benchmarks, underscoring its superior effectiveness and explainability.
Key Contributions
- Novel encoder-decoder architecture repurposing a pretrained forgery detector as an 'artifact encoder' and a denoising diffusion model as an 'artifact decoder' for fine-grained localization
- Joint face forgery detection and artifact localization in a single framework, fusing localization maps with high-level semantic features to improve detection accuracy
- State-of-the-art performance across multiple deepfake detection benchmarks with enhanced model explainability
🛡️ Threat Analysis
Deepfake/face forgery detection is a canonical ML09 task (AI-generated content detection). DiffusionFF proposes a novel detection architecture — not merely applying existing methods — that simultaneously classifies manipulated images and produces fine-grained localization maps of forgery artifacts.