ML Security PapersReconciling Communication Compression and Byzantine-Robustness in Distributed Learning
Diksha Gupta , Antonio Honsell , Chuan Xu , Nirupam Gupta , Giovanni Neglia
Published on arXiv
2508.17129
Data Poisoning Attack
OWASP ML Top 10 — ML02
Key Finding
RoSDHB achieves stronger Byzantine robustness than Byz-DASHA-PAGE while using less memory and fewer bits per client under the standard (G,B)-gradient dissimilarity model.
RoSDHB (Robust Sparsified Distributed Heavy-Ball)
Novel technique introduced
Distributed learning enables scalable model training over decentralized data, but remains hindered by Byzantine faults and high communication costs. While both challenges have been studied extensively in isolation, their interplay has received limited attention. Prior work has shown that naively combining communication compression with Byzantine-robust aggregation can severely weaken resilience to faulty nodes. The current state-of-the-art, Byz-DASHA-PAGE, leverages a momentum-based variance reduction scheme to counteract the negative effect of compression noise on Byzantine robustness. In this work, we introduce RoSDHB, a new algorithm that integrates classical Polyak momentum with a coordinated compression strategy. Theoretically, RoSDHB matches the convergence guarantees of Byz-DASHA-PAGE under the standard $(G,B)$-gradient dissimilarity model, while relying on milder assumptions and requiring less memory and communication per client. Empirically, RoSDHB demonstrates stronger robustness while achieving substantial communication savings compared to Byz-DASHA-PAGE.
Key Contributions
- RoSDHB algorithm integrating classical Polyak (Heavy-Ball) momentum with a coordinated sparsification compression scheme for Byzantine-robust distributed learning
- Theoretical convergence guarantees matching Byz-DASHA-PAGE under the (G,B)-gradient dissimilarity model while requiring milder assumptions and less memory/communication per client
- Empirical demonstration of stronger Byzantine robustness and substantial communication savings over the prior state-of-the-art Byz-DASHA-PAGE
🛡️ Threat Analysis
Proposes RoSDHB, a Byzantine-robust aggregation protocol for distributed learning that defends against malicious clients sending arbitrary model updates to corrupt global model training — a direct Byzantine-fault-tolerant FL defense explicitly listed under ML02.