Protection against Source Inference Attacks in Federated Learning

Federated Learning (FL) was initially proposed as a privacy-preserving machine learning paradigm. However, FL has been shown to be susceptible to a series of privacy attacks. Recently, there has been concern about the Source Inference Attack (SIA), where an honest-but-curious central server attempts to identify exactly which client owns a given data point which was used in the training phase. Alarmingly, standard gradient obfuscation techniques with Differential Privacy have been shown to be ineffective against SIAs, at least without severely diminishing the accuracy. In this work, we propose a defense against SIAs within the widely studied shuffle model of FL, where an honest shuffler acts as an intermediary between the clients and the server. First, we demonstrate that standard naive shuffling alone is insufficient to prevent SIAs. To effectively defend against SIAs, shuffling needs to be applied at a more granular level; we propose a novel combination of parameter-level shuffling with the residue number system (RNS). Our approach provides robust protection against SIAs without affecting the accuracy of the joint model and can be seamlessly integrated into other privacy protection mechanisms. We conduct experiments on a series of models and datasets, confirming that standard shuffling approaches fail to prevent SIAs and that, in contrast, our proposed method reduce the attack's accuracy to the level of random guessing.

Key Contributions

Novel reconstruction attacks showing that naive model-level, layer-level, and parameter-level shuffling alone cannot prevent SIAs by remapping shuffled values back to their original clients
A defense combining parameter-level shuffling with the Residue Number System (RNS) that reduces SIA accuracy to random guessing without degrading joint model accuracy
Empirical demonstration that standard defenses (DP, regularization, data reconstruction defenses) are ineffective against SIAs in FL

🛡️ Threat Analysis

Membership Inference Attack

The Source Inference Attack (SIA) is a finer-grained extension of membership inference: instead of asking whether a data point is in ANY client's training set, it asks which specific client owns it. The paper defends against this attribution-level membership inference by reducing SIA accuracy to random guessing using parameter-level shuffling with RNS.