ML Security PapersToward Efficient Inference Attacks: Shadow Model Sharing via Mixture-of-Experts
Li Bai 1,2, Qingqing Ye 1,2, Xinwei Zhang 1, Sen Zhang 1, Zi Liang 1, Jianliang Xu 3, Haibo Hu 1,2
1 The Hong Kong Polytechnic University
2 PolyU Research Centre for Privacy and Security Technologies in Future Smart Systems
Published on arXiv
2510.13451
Membership Inference Attack
OWASP ML Top 10 — ML04
Key Finding
SHAPOOL significantly reduces the computational cost of shadow model construction while maintaining attack performance comparable to using fully independent shadow models across multiple membership inference attack settings.
SHAPOOL
Novel technique introduced
Machine learning models are often vulnerable to inference attacks that expose sensitive information from their training data. Shadow model technique is commonly employed in such attacks, such as membership inference. However, the need for a large number of shadow models leads to high computational costs, limiting their practical applicability. Such inefficiency mainly stems from the independent training and use of these shadow models. To address this issue, we present a novel shadow pool training framework SHAPOOL, which constructs multiple shared models and trains them jointly within a single process. In particular, we leverage the Mixture-of-Experts mechanism as the shadow pool to interconnect individual models, enabling them to share some sub-networks and thereby improving efficiency. To ensure the shared models closely resemble independent models and serve as effective substitutes, we introduce three novel modules: path-choice routing, pathway regularization, and pathway alignment. These modules guarantee random data allocation for pathway learning, promote diversity among shared models, and maintain consistency with target models. We evaluate SHAPOOL in the context of various membership inference attacks and show that it significantly reduces the computational cost of shadow model construction while maintaining comparable attack performance.
Key Contributions
- SHAPOOL framework that jointly trains a pool of shadow models within a single process using Mixture-of-Experts, replacing independent training of many separate models
- Three novel modules — path-choice routing, pathway regularization, and pathway alignment — that ensure shared sub-networks behave like independent shadow models while remaining aligned with the target model
- Demonstrated significant reduction in computational cost for shadow model construction with comparable membership inference attack performance across multiple MIA settings
🛡️ Threat Analysis
The paper directly targets membership inference attacks, proposing SHAPOOL to reduce the high computational cost of training many shadow models — which are the standard vehicle for MIA. All three novel modules (path-choice routing, pathway regularization, pathway alignment) are designed to make shadow models more effective substitutes for the target model in an MIA context.