SPEAR++: Scaling Gradient Inversion via Sparsely-Used Dictionary Learning

Federated Learning has seen an increased deployment in real-world scenarios recently, as it enables the distributed training of machine learning models without explicit data sharing between individual clients. Yet, the introduction of the so-called gradient inversion attacks has fundamentally challenged its privacy-preserving properties. Unfortunately, as these attacks mostly rely on direct data optimization without any formal guarantees, the vulnerability of real-world systems remains in dispute and requires tedious testing for each new federated deployment. To overcome these issues, recently the SPEAR attack was introduced, which is based on a theoretical analysis of the gradients of linear layers with ReLU activations. While SPEAR is an important theoretical breakthrough, the attack's practicality was severely limited by its exponential runtime in the batch size b. In this work, we fill this gap by applying State-of-the-Art techniques from Sparsely-Used Dictionary Learning to make the problem of gradient inversion on linear layers with ReLU activations tractable. Our experiments demonstrate that our new attack, SPEAR++, retains all desirable properties of SPEAR, such as robustness to DP noise and FedAvg aggregation, while being applicable to 10x bigger batch sizes.

Key Contributions

• Applies Sparsely-Used Dictionary Learning to gradient inversion on linear layers with ReLU activations, reducing exponential runtime to tractable complexity
• Demonstrates SPEAR++ scales gradient inversion to batch sizes 10x larger than the original SPEAR attack while preserving theoretical guarantees
• Maintains SPEAR's robustness properties against DP noise and FedAvg aggregation in the scaled setting

🛡️ Threat Analysis

Details

Similar Papers