MSCT: Differential Cross-Modal Attention for Deepfake Detection

Audio-visual deepfake detection typically employs a complementary multi-modal model to check the forgery traces in the video. These methods primarily extract forgery traces through audio-visual alignment, which results from the inconsistency between audio and video modalities. However, the traditional multi-modal forgery detection method has the problem of insufficient feature extraction and modal alignment deviation. To address this, we propose a multi-scale cross-modal transformer encoder (MSCT) for deepfake detection. Our approach includes a multi-scale self-attention to integrate the features of adjacent embeddings and a differential cross-modal attention to fuse multi-modal features. Our experiments demonstrate competitive performance on the FakeAVCeleb dataset, validating the effectiveness of the proposed structure.

Key Contributions

• Multi-scale self-attention module to extract temporal features across adjacent video frames
• Differential cross-modal attention module that improves compatibility between cross-modal attention and alignment loss by focusing on audio-visual inconsistencies in fake videos
• Achieves competitive performance on FakeAVCeleb dataset for audio-visual deepfake detection

🛡️ Threat Analysis

Details

Similar Papers