Hierarchical Deep Fusion Framework for Multi-dimensional Facial Forgery Detection - The 2024 Global Deepfake Image Detection Challenge

The proliferation of sophisticated deepfake technology poses significant challenges to digital security and authenticity. Detecting these forgeries, especially across a wide spectrum of manipulation techniques, requires robust and generalized models. This paper introduces the Hierarchical Deep Fusion Framework (HDFF), an ensemble-based deep learning architecture designed for high-performance facial forgery detection. Our framework integrates four diverse pre-trained sub-models, Swin-MLP, CoAtNet, EfficientNetV2, and DaViT, which are meticulously fine-tuned through a multi-stage process on the MultiFFDI dataset. By concatenating the feature representations from these specialized models and training a final classifier layer, HDFF effectively leverages their collective strengths. This approach achieved a final score of 0.96852 on the competition's private leaderboard, securing the 20th position out of 184 teams, demonstrating the efficacy of hierarchical fusion for complex image classification tasks.

Key Contributions

• HDFF ensemble architecture integrating Swin-MLP, CoAtNet, EfficientNetV2, and DaViT via feature concatenation and a fused FC classifier for deepfake detection
• Multi-stage training strategy: ImageNet-pretrained initialization → frozen-backbone head fine-tuning → full end-to-end fine-tuning
• Competitive validation on MultiFFDI (50+ forgery methods), achieving 0.96852 score and 20th place out of 184 teams

🛡️ Threat Analysis

Details

Similar Papers