VocalBridge: Latent Diffusion-Bridge Purification for Defeating Perturbation-Based Voiceprint Defenses

The rapid advancement of speech synthesis technologies, including text-to-speech (TTS) and voice conversion (VC), has intensified security and privacy concerns related to voice cloning. Recent defenses attempt to prevent unauthorized cloning by embedding protective perturbations into speech to obscure speaker identity while maintaining intelligibility. However, adversaries can apply advanced purification techniques to remove these perturbations, recover authentic acoustic characteristics, and regenerate cloneable voices. Despite the growing realism of such attacks, the robustness of existing defenses under adaptive purification remains insufficiently studied. Most existing purification methods are designed to counter adversarial noise in automatic speech recognition (ASR) systems rather than speaker verification or voice cloning pipelines. As a result, they fail to suppress the fine-grained acoustic cues that define speaker identity and are often ineffective against speaker verification attacks (SVA). To address these limitations, we propose Diffusion-Bridge (VocalBridge), a purification framework that learns a latent mapping from perturbed to clean speech in the EnCodec latent space. Using a time-conditioned 1D U-Net with a cosine noise schedule, the model enables efficient, transcript-free purification while preserving speaker-discriminative structure. We further introduce a Whisper-guided phoneme variant that incorporates lightweight temporal guidance without requiring ground-truth transcripts. Experimental results show that our approach consistently outperforms existing purification methods in recovering cloneable voices from protected speech. Our findings demonstrate the fragility of current perturbation-based defenses and highlight the need for more robust protection mechanisms against evolving voice-cloning and speaker verification threats.

Key Contributions

VocalBridge: a latent diffusion-bridge purification framework trained to map perturbed speech to clean speech in the EnCodec latent space using a time-conditioned 1D U-Net with a cosine noise schedule
Whisper-guided phoneme variant that provides lightweight temporal acoustic guidance for transcript-free purification
Empirical demonstration of the fragility of current perturbation-based voiceprint defenses (e.g., AntiFake-style protections) against adaptive purification attacks targeting speaker verification and voice cloning pipelines

🛡️ Threat Analysis

Output Integrity Attack

VocalBridge attacks perturbation-based voiceprint protection schemes (protective adversarial noise embedded in speech to prevent cloning), analogous to defeating anti-deepfake or style-transfer protections on images. Removing/defeating these protective perturbations is an ML09 attack on content integrity and protection schemes, not a standard adversarial example attack.