ML Security PapersVoxGuard: Evaluating User and Attribute Privacy in Speech via Membership Inference Attacks
Efthymios Tsaprazlis , Thanathai Lertpetchpun , Tiantian Feng , Sai Praneeth Karimireddy , Shrikanth Narayanan
Published on arXiv
2509.18413
Membership Inference Attack
OWASP ML Top 10 — ML04
Key Finding
Informed adversaries using fine-tuned models and max-similarity scoring achieve orders-of-magnitude higher attack success at low-FPR than EER suggests, and simple transparent attacks recover gender and accent with near-perfect accuracy after anonymization.
VoxGuard
Novel technique introduced
Voice anonymization aims to conceal speaker identity and attributes while preserving intelligibility, but current evaluations rely almost exclusively on Equal Error Rate (EER) that obscures whether adversaries can mount high-precision attacks. We argue that privacy should instead be evaluated in the low false-positive rate (FPR) regime, where even a small number of successful identifications constitutes a meaningful breach. To this end, we introduce VoxGuard, a framework grounded in differential privacy and membership inference that formalizes two complementary notions: User Privacy, preventing speaker re-identification, and Attribute Privacy, protecting sensitive traits such as gender and accent. Across synthetic and real datasets, we find that informed adversaries, especially those using fine-tuned models and max-similarity scoring, achieve orders-of-magnitude stronger attacks at low-FPR despite similar EER. For attributes, we show that simple transparent attacks recover gender and accent with near-perfect accuracy even after anonymization. Our results demonstrate that EER substantially underestimates leakage, highlighting the need for low-FPR evaluation, and recommend VoxGuard as a benchmark for evaluating privacy leakage.
Key Contributions
- Reframes speech privacy evaluation as a membership inference problem with two formal notions: User Privacy (speaker re-identification) and Attribute Privacy (gender, accent inference), grounded in differential privacy
- Argues that EER is an inadequate privacy metric and privacy evaluation should focus on the low-FPR regime to capture worst-case leakage; shows informed adversaries achieve orders-of-magnitude stronger attacks at low-FPR despite comparable EER
- Demonstrates that transparent attribute attacks recover gender and accent with near-perfect accuracy even after voice anonymization, exposing a critical unaddressed leakage vector
🛡️ Threat Analysis
The paper's primary contribution is formalizing and evaluating membership inference attacks against voice anonymization systems — both User Privacy (speaker re-identification) and Attribute Privacy (gender, accent) are framed explicitly as MIA problems evaluated in the low-FPR regime.