ML Security PapersImproving Out-of-Domain Audio Deepfake Detection via Layer Selection and Fusion of SSL-Based Countermeasures
Pierre Serrano , Raphaël Duroselle , Florian Angulo , Jean-François Bonastre , Olivier Boeffard
Published on arXiv
2509.12003
Output Integrity Attack
OWASP ML Top 10 — ML09
Key Finding
Optimal single-layer selection reduces parameters by up to 80% with competitive OOD performance; score-level fusion of multiple SSL encoders (WavLM, BEATs, etc.) significantly improves generalization to unseen attack types.
Layer Selection + Score-Level Fusion for SSL-based Countermeasures
Novel technique introduced
Audio deepfake detection systems based on frozen pre-trained self-supervised learning (SSL) encoders show a high level of performance when combined with layer-weighted pooling methods, such as multi-head factorized attentive pooling (MHFA). However, they still struggle to generalize to out-of-domain (OOD) conditions. We tackle this problem by studying the behavior of six different pre-trained SSLs, on four different test corpora. We perform a layer-by-layer analysis to determine which layers contribute most. Next, we study the pooling head, comparing a strategy based on a single layer with automatic selection via MHFA. We observed that selecting the best layer gave very good results, while reducing system parameters by up to 80%. A wide variation in performance as a function of test corpus and SSL model is also observed, showing that the pre-training strategy of the encoder plays a role. Finally, score-level fusion of several encoders improved generalization to OOD attacks.
Key Contributions
- Layer-by-layer analysis of six SSL encoders to identify which intermediate layers best generalize to out-of-domain audio deepfake detection
- Single-best-layer selection strategy that matches MHFA performance while reducing system parameters by up to 80%
- Score-level fusion of diverse SSL encoders that improves OOD generalization across four test corpora
🛡️ Threat Analysis
Directly addresses detection of AI-generated audio content (audio deepfakes); proposes novel forensic techniques — layer-wise analysis, best-layer selection, and score-level SSL fusion — to improve authentication of audio output integrity across OOD conditions.