Improving Generalization in Deepfake Detection with Face Foundation Models and Metric Learning

The increasing realism and accessibility of deepfakes have raised critical concerns about media authenticity and information integrity. Despite recent advances, deepfake detection models often struggle to generalize beyond their training distributions, particularly when applied to media content found in the wild. In this work, we present a robust video deepfake detection framework with strong generalization that takes advantage of the rich facial representations learned by face foundation models. Our method is built on top of FSFM, a self-supervised model trained on real face data, and is further fine-tuned using an ensemble of deepfake datasets spanning both face-swapping and face-reenactment manipulations. To enhance discriminative power, we incorporate triplet loss variants during training, guiding the model to produce more separable embeddings between real and fake samples. Additionally, we explore attribution-based supervision schemes, where deepfakes are categorized by manipulation type or source dataset, to assess their impact on generalization. Extensive experiments across diverse evaluation benchmarks demonstrate the effectiveness of our approach, especially in challenging real-world scenarios.

Key Contributions

• Deepfake detection framework built on FSFM (a self-supervised face foundation model) fine-tuned across diverse face-swapping and face-reenactment datasets for cross-dataset generalization
• Integration of triplet loss variants to produce more separable embeddings between real and fake samples, improving discriminative power
• Attribution-based supervision schemes that categorize deepfakes by manipulation type or source dataset to assess their impact on generalization

🛡️ Threat Analysis

Details

Similar Papers