The Model's Language Matters: A Comparative Privacy Analysis of LLMs

Large Language Models (LLMs) are increasingly deployed across multilingual applications that handle sensitive data, yet their scale and linguistic variability introduce major privacy risks. Mostly evaluated for English, this paper investigates how language structure affects privacy leakage in LLMs trained on English, Spanish, French, and Italian medical corpora. We quantify six linguistic indicators and evaluate three attack vectors: extraction, counterfactual memorization, and membership inference. Results show that privacy vulnerability scales with linguistic redundancy and tokenization granularity: Italian exhibits the strongest leakage, while English shows higher membership separability. In contrast, French and Spanish display greater resilience due to higher morphological complexity. Overall, our findings provide the first quantitative evidence that language matters in privacy leakage, underscoring the need for language-aware privacy-preserving mechanisms in LLM deployments.

Key Contributions

First quantitative cross-lingual analysis linking linguistic properties (redundancy, morphological complexity, tokenization granularity) to LLM privacy vulnerability
Evaluation of extraction, counterfactual memorization, and membership inference attacks across English, Spanish, French, and Italian medical LLMs
Evidence that Italian exhibits the strongest leakage while French and Spanish show greater resilience due to morphological richness

🛡️ Threat Analysis

Model Inversion Attack

Extraction and counterfactual memorization attacks are directly evaluated as methods to recover private training data from fine-tuned LLMs across four languages.

Membership Inference Attack

Membership inference attacks are explicitly evaluated, showing English fine-tuned models exhibit the clearest in/out sample separability.