ML Security PapersHarmonizing Multi-Objective LLM Unlearning via Unified Domain Representation and Bidirectional Logit Distillation
Yisheng Zhong 1, Sijia Liu 2, Zhuangdi Zhu 1
Published on arXiv
2604.15482
Model Inversion Attack
OWASP ML Top 10 — ML03
Prompt Injection
OWASP LLM Top 10 — LLM01
Sensitive Information Disclosure
OWASP LLM Top 10 — LLM06
Key Finding
Reduces prefilling attack success rates to 16.0% while preserving general utility and avoiding over-refusal in adjacent domains
Dual Top-K Push-Pull Distillation
Novel technique introduced
Large Language Models (LLMs) unlearning is crucial for removing hazardous or privacy-leaking information from the model. Practical LLM unlearning demands satisfying multiple challenging objectives simultaneously: removing undesirable knowledge, preserving general utility, avoiding over-refusal of neighboring concepts, and, crucially, ensuring robustness against adversarial probing attacks. However, existing unlearning methods primarily focus on a limited subset of these goals, typically unlearning efficacy and utility preservation while overlooking robustness and boundary behaviors. Naively extending these methods to multi-objective settings may lead to unlearning task interference. We propose a novel multi-objective unlearning framework that harmonizes multiple unlearning objectives through a data and optimization co-design: We standardize training corpora into a unified data representation to reduce the domain gap, and then introduce a bidirectional distillation method that simultaneously elicits desired behavior from a context-instructed teacher while suppressing undesirable behavior in the student model. Theoretical and empirical analyses show that our method aligns domain distributions and converts seemingly irrelevant unlearning tasks into cooperative optimization. Evaluation demonstrates state-of-the-art performance, which enables balanced and reliable unlearning across diverse, challenging requirements.
Key Contributions
- Dual Top-K Push-Pull Logit Distillation that bidirectionally aligns student and teacher logits to remove undesirable knowledge while preserving utility
- Data format standardization that projects all training corpora into unified QA representation to align marginal distributions and reduce task interference
- Multi-objective framework that simultaneously optimizes forgetting target knowledge, preventing over-refusal, and resisting adversarial probing attacks
🛡️ Threat Analysis
The paper addresses removing privacy-leaking and hazardous information from LLMs with explicit robustness evaluation against adversarial probing attacks (prefilling attacks) that attempt to extract unlearned knowledge. The threat model includes an adversary trying to recover supposedly-removed knowledge through adversarial prompts, making this a data extraction defense.