ML Security PapersStatistical Roughness-Informed Machine Unlearning
Mohammad Partohaghighi 1, Roummel Marcia 1, Bruce J. West 2, YangQuan Chen 1
Published on arXiv
2602.09304
Membership Inference Attack
OWASP ML Top 10 — ML04
Key Finding
SRAGU improves stability under hard/adversarial forget-set deletions by steering unlearning updates toward spectrally stable layers, improving behavioral alignment to gold-retrained reference models.
SRAGU (Statistical-Roughness Adaptive Gradient Unlearning)
Novel technique introduced
Machine unlearning aims to remove the influence of a designated forget set from a trained model while preserving utility on the retained data. In modern deep networks, approximate unlearning frequently fails under large or adversarial deletions due to pronounced layer-wise heterogeneity: some layers exhibit stable, well-regularized representations while others are brittle, undertrained, or overfit, so naive update allocation can trigger catastrophic forgetting or unstable dynamics. We propose Statistical-Roughness Adaptive Gradient Unlearning (SRAGU), a mechanism-first unlearning algorithm that reallocates unlearning updates using layer-wise statistical roughness operationalized via heavy-tailed spectral diagnostics of layer weight matrices. Starting from an Adaptive Gradient Unlearning (AGU) sensitivity signal computed on the forget set, SRAGU estimates a WeightWatcher-style heavy-tailed exponent for each layer, maps it to a bounded spectral stability weight, and uses this stability signal to spectrally reweight the AGU sensitivities before applying the same minibatch update form. This concentrates unlearning motion in spectrally stable layers while damping updates in unstable or overfit layers, improving stability under hard deletions. We evaluate unlearning via behavioral alignment to a gold retrained reference model trained from scratch on the retained data, using empirical prediction-divergence and KL-to-gold proxies on a forget-focused query set; we additionally report membership inference auditing as a complementary leakage signal, treating forget-set points as should-be-forgotten members during evaluation.
Key Contributions
- Identifies layer-wise heterogeneity as a concrete failure mode of sensitivity-driven unlearning algorithms
- Proposes SRAGU, which reweights AGU unlearning updates using heavy-tailed spectral exponents (WeightWatcher-style) to concentrate updates in spectrally stable layers and dampen them in brittle/overfit layers
- Evaluates unlearning fidelity via KL divergence to a gold retrained model and membership inference auditing as a privacy leakage proxy
🛡️ Threat Analysis
The paper explicitly reports membership inference auditing as a complementary leakage signal — treating forget-set points as 'should-be-forgotten members' — placing the evaluation squarely in an adversarial membership inference threat model. The introduction also frames unlearning's importance in terms of preventing membership inference and training data extraction.