Physics-Guided Deepfake Detection for Voice Authentication Systems

Voice authentication systems deployed at the network edge face dual threats: a) sophisticated deepfake synthesis attacks and b) control-plane poisoning in distributed federated learning protocols. We present a framework coupling physics-guided deepfake detection with uncertainty-aware in edge learning. The framework fuses interpretable physics features modeling vocal tract dynamics with representations coming from a self-supervised learning module. The representations are then processed via a Multi-Modal Ensemble Architecture, followed by a Bayesian ensemble providing uncertainty estimates. Incorporating physics-based characteristics evaluations and uncertainty estimates of audio samples allows our proposed framework to remain robust to both advanced deepfake attacks and sophisticated control-plane poisoning, addressing the complete threat model for networked voice authentication.

Key Contributions

Multi-modal ensemble fusing interpretable physics features (vocal tract dynamics) with self-supervised learning representations for generalizable audio deepfake detection
Bayesian uncertainty quantification that provides calibrated trust signals for both detection reliability and adversarial client screening in federated edge deployments
Trust-based FL aggregation protocol that exploits detector uncertainty estimates as domain-specific metrics to screen malicious parameter updates without exposing raw user data

🛡️ Threat Analysis

Data Poisoning Attack

The paper explicitly defends against control-plane poisoning in federated learning, where adversarial clients submit malicious parameter updates to degrade the global detection model. The proposed trust-based aggregation protocol using calibrated uncertainty estimates is a direct defense against Byzantine FL attacks — exactly ML02's scope for federated learning poisoning defenses.

Output Integrity Attack

The core contribution is a novel audio deepfake detection architecture — detecting AI-synthesized speech using physics-guided vocal tract features, SSL representations, and a Bayesian ensemble. This is AI-generated content detection with a novel detection methodology, not merely applying existing methods to a domain.