ML Security PapersCODED-SMOOTHING: Coding Theory Helps Generalization
Parsa Moradi, Tayyebeh Jahaninezhad, Mohammad Ali Maddah-Ali · University of Minnesota · Technical University Berlin
Regularization module from coded computing theory that defends against gradient-based adversarial attacks by enforcing smooth model representations
We introduce the coded-smoothing module, which can be seamlessly integrated into standard training pipelines, both supervised and unsupervised, to regularize learning and improve generalization with minimal computational overhead. In addition, it can be incorporated into the inference pipeline to randomize the model and enhance robustness against adversarial perturbations. The design of coded-smoothing is inspired by general coded computing, a paradigm originally developed to mitigate straggler and adversarial failures in distributed computing by processing linear combinations of the data rather than the raw inputs. Building on this principle, we adapt coded computing to machine learning by designing an efficient and effective regularization mechanism that encourages smoother representations and more generalizable solutions. Extensive experiments on both supervised and unsupervised tasks demonstrate that coded-smoothing consistently improves generalization and achieves state-of-the-art robustness against gradient-based adversarial attacks.