ML Security PapersLBFTI: Layer-Based Facial Template Inversion for Identity-Preserving Fine-Grained Face Reconstruction
Zixuan Shen , Zhihua Xia , Kaikai Gan , Peipeng Yu
Published on arXiv
2604.18358
Model Inversion Attack
OWASP ML Top 10 — ML03
Key Finding
Achieves 25.3% improvement in TAR over SOTA methods in Type-II attack scenarios, with 33.5% reduction in FAPD and 2.9% reduction in FAPC; 85% of human participants judge LBFTI reconstructions as more similar to originals
LBFTI
Novel technique introduced
In face recognition systems, facial templates are widely adopted for identity authentication due to their compliance with the data minimization principle. However, facial template inversion technologies have posed a severe privacy leakage risk by enabling face reconstruction from templates. This paper proposes a Layer-Based Facial Template Inversion (LBFTI) method to reconstruct identity-preserving fine-grained face images. Our scheme decomposes face images into three layers: foreground layers (including eyebrows, eyes, nose, and mouth), midground layers (skin), and background layers (other parts). LBFTI leverages dedicated generators to produce these layers, adopting a rigorous three-stage training strategy: (1) independent refined generation of foreground and midground layers, (2) fusion of foreground and midground layers with template secondary injection to produce complete panoramic face images with background layers, and (3) joint fine-tuning of all modules to optimize inter-layer coordination and identity consistency. Experiments demonstrate that our LBFTI not only outperforms state-of-the-art methods in machine authentication performance, with a 25.3% improvement in TAR, but also achieves better similarity in human perception, as validated by both quantitative metrics and a questionnaire survey.
Key Contributions
- Layer-based facial template inversion framework decomposing face reconstruction into foreground (facial features), midground (skin), and background layers with dedicated generators
- Three-stage training strategy: (1) independent generation of foreground/midground, (2) fusion with template injection for panoramic faces, (3) joint fine-tuning for identity consistency
- Human-perception-oriented evaluation metrics: Facial Attribute Pixel Deviation (FAPD) and Facial Attribute Perceptual Consistency (FAPC) for fine-grained facial attribute assessment
🛡️ Threat Analysis
Core contribution is a model inversion attack that reconstructs private training data (face images) from facial recognition templates. The adversary reverse-engineers the FR model to recover detailed face images from compact template representations, which is the definition of model inversion.