ML Security PapersLipschitz verification of neural networks through training
Simon Kuang 1, Yuezhu Xu 2, S. Sivaranjani 2, Xinfan Lin 1
Published on arXiv
2603.28113
Input Manipulation Attack
OWASP ML Top 10 — ML01
Key Finding
Achieves Lipschitz bounds within 10% of ground truth on MNIST, orders of magnitude lower than comparable certified training methods
Lipschitz-constrained training via trivial bound penalization
Novel technique introduced
The global Lipschitz constant of a neural network governs both adversarial robustness and generalization. Conventional approaches to ``certified training" typically follow a train-then-verify paradigm: they train a network and then attempt to bound its Lipschitz constant. Because the efficient ``trivial bound" (the product of the layerwise Lipschitz constants) is exponentially loose for arbitrary networks, these approaches must rely on computationally expensive techniques such as semidefinite programming, mixed-integer programming, or branch-and-bound. We propose a different paradigm: rather than designing complex verifiers for arbitrary networks, we design networks to be verifiable by the fast trivial bound. We show that directly penalizing the trivial bound during training forces it to become tight, thereby effectively regularizing the true Lipschitz constant. To achieve this, we identify three structural obstructions to a tight trivial bound (dead neurons, bias terms, and ill-conditioned weights) and introduce architectural mitigations, including a novel notion of norm-saturating polyactivations and bias-free sinusoidal layers. Our approach avoids the runtime complexity of advanced verification while achieving strong results: we train robust networks on MNIST with Lipschitz bounds that are small (orders of magnitude lower than comparable works) and tight (within 10% of the ground truth). The experimental results validate the theoretical guarantees, support the proposed mechanisms, and extend empirically to diverse activations and non-Euclidean norms.
Key Contributions
- Novel training paradigm that penalizes the trivial Lipschitz bound directly, making it tight and avoiding expensive post-hoc verification
- Identifies and mitigates three structural obstructions to tight bounds: dead neurons, bias terms, and ill-conditioned weights
- Introduces norm-saturating polyactivations and bias-free sinusoidal layers to enable efficient certified robustness
🛡️ Threat Analysis
The paper addresses adversarial robustness by controlling the Lipschitz constant, which bounds the model's sensitivity to input perturbations. The Lipschitz constant directly governs resistance to adversarial examples. The paper trains networks with provable robustness guarantees against input manipulation attacks.