ML Security PapersSecret Stealing Attacks on Local LLM Fine-Tuning through Supply-Chain Model Code Backdoors
Zi Li , Tian Zhou , Wenze Li , Jingyu Hua , Yunlong Mao , Sheng Zhong
Published on arXiv
2604.27426
AI Supply Chain Attacks
OWASP ML Top 10 — ML06
Model Inversion Attack
OWASP ML Top 10 — ML03
Model Poisoning
OWASP ML Top 10 — ML10
Sensitive Information Disclosure
OWASP LLM Top 10 — LLM06
Key Finding
Achieves over 98% strict attack success rate in stealing API keys and credentials while bypassing DP-SGD and code auditing defenses
Supply-Chain Model Code Backdoor
Novel technique introduced
Local fine-tuning datasets routinely contain sensitive secrets such as API keys, personal identifiers, and financial records. Although ''local offline fine-tuning'' is often viewed as a privacy boundary, we reveal that compromised model code is sufficient to steal them. Current passive pretrained-weight poisoning attacks, while effective for natural language, fundamentally fail to capture such sparse high-entropy targets due to their reliance on probabilistic semantic prefixes. To bridge this gap, we identify and exploit a practical but overlooked supply-chain vector -- model code camouflaged as standard architectural definitions -- to realize a paradigm shift from passive weight poisoning to active execution hijacking. We introduce a deterministic full-chain memorization mechanism: it locks onto token-level secrets in dynamic computation flows via online tensor-rule matching, and leverages value-gradient decoupling to stealthily inject attack gradients, overcoming gradient drowning to force model memorization. Furthermore, we achieve, for the first time, attacker-verifiable secret stealing through black-box queries that precisely distinguishes true leakage from hallucination. Experiments demonstrate that our method achieves over 98\% Strict ASR without compromising the primary task, and can effectively bypass defense measures including DP-SGD, semantic auditing, and code auditing.
Key Contributions
- First supply-chain attack exploiting malicious model code (not just weights) to steal high-entropy secrets during fine-tuning
- Deterministic memorization mechanism using online tensor-rule matching and value-gradient decoupling to overcome gradient drowning
- Attacker-verifiable black-box extraction method that distinguishes true leakage from hallucination
🛡️ Threat Analysis
Attack goal is extracting private training data (API keys, SSNs, financial records) from fine-tuned models via black-box queries.
Primary attack vector is compromised model implementation code distributed via supply chain (modeling_xxx.py files with malicious logic camouflaged as architectural definitions).
Embeds backdoors that activate during fine-tuning to force memorization of target secrets through gradient manipulation.