ML Security PapersSIDeR: Semantic Identity Decoupling for Unrestricted Face Privacy
Zhuosen Bao 1, Xia Du 1, Zheng Lin 2, Jizhe Zhou 3, Zihan Fang 4, Jiening Wu 5, Yuxin Zhang 6, Zhe Chen 6, Chi-Man Pun 7, Wei Ni 8,9, Jun Luo 10
1 Xiamen University of Technology
4 City University of Hong Kong
8 CSIRO
Published on arXiv
2602.04994
Input Manipulation Attack
OWASP ML Top 10 — ML01
Key Finding
Achieves 99% black-box attack success rate against face recognition systems and surpasses baseline restoration quality by 41.28% in PSNR on CelebA-HQ and FFHQ.
SIDeR
Novel technique introduced
With the deep integration of facial recognition into online banking, identity verification, and other networked services, achieving effective decoupling of identity information from visual representations during image storage and transmission has become a critical challenge for privacy protection. To address this issue, we propose SIDeR, a Semantic decoupling-driven framework for unrestricted face privacy protection. SIDeR decomposes a facial image into a machine-recognizable identity feature vector and a visually perceptible semantic appearance component. By leveraging semantic-guided recomposition in the latent space of a diffusion model, it generates visually anonymous adversarial faces while maintaining machine-level identity consistency. The framework incorporates momentum-driven unrestricted perturbation optimization and a semantic-visual balancing factor to synthesize multiple visually diverse, highly natural adversarial samples. Furthermore, for authorized access, the protected image can be restored to its original form when the correct password is provided. Extensive experiments on the CelebA-HQ and FFHQ datasets demonstrate that SIDeR achieves a 99% attack success rate in black-box scenarios and outperforms baseline methods by 41.28% in PSNR-based restoration quality.
Key Contributions
- SIDeR framework that decomposes facial images into identity feature vectors and semantic appearance components, then recomposes them in diffusion model latent space to generate visually anonymous adversarial faces
- Momentum-driven unrestricted perturbation optimization with a semantic-visual balancing factor for diverse, natural-looking adversarial samples
- Password-controlled reversible restoration that reconstructs the original face with high fidelity (41.28% PSNR improvement over baselines) for authorized access
🛡️ Threat Analysis
SIDeR crafts adversarial facial images that cause face recognition models to fail identification at inference time — this is a classic evasion/input manipulation attack. The momentum-driven unrestricted perturbation optimization and diffusion-based semantic recomposition are adversarial example generation techniques specifically designed to defeat ML-based face recognition systems in black-box settings.