ML Security PapersProtego: User-Centric Pose-Invariant Privacy Protection Against Face Recognition-Induced Digital Footprint Exposure
Ziling Wang , Shuya Yang , Jialin Lu , Ka-Ho Chow
Published on arXiv
2508.02034
Input Manipulation Attack
OWASP ML Top 10 — ML01
Key Finding
Protego reduces FR-based retrieval accuracy by at least 2x compared to existing methods while maintaining visual naturalness across poses and video frames
Protego
Novel technique introduced
Face recognition (FR) technologies are increasingly used to power large-scale image retrieval systems, raising serious privacy concerns. Services like Clearview AI and PimEyes allow anyone to upload a facial photo and retrieve a large amount of online content associated with that person. This not only enables identity inference but also exposes their digital footprint, such as social media activity, private photos, and news reports, often without their consent. In response to this emerging threat, we propose Protego, a user-centric privacy protection method that safeguards facial images from such retrieval-based privacy intrusions. Protego encapsulates a user's 3D facial signatures into a pose-invariant 2D representation, which is dynamically deformed into a natural-looking 3D mask tailored to the pose and expression of any facial image of the user, and applied prior to online sharing. Motivated by a critical limitation of existing methods, Protego amplifies the sensitivity of FR models so that protected images cannot be matched even among themselves. Experiments show that Protego significantly reduces retrieval accuracy across a wide range of black-box FR models and performs at least 2x better than existing methods. It also offers unprecedented visual coherence, particularly in video settings where consistency and natural appearance are essential. Overall, Protego contributes to the fight against the misuse of FR for mass surveillance and unsolicited identity tracing.
Key Contributions
- Pose-invariant Privacy Protection Texture (PPT) that encodes 3D facial signatures into a 2D representation dynamically deformed to any head pose or expression
- Novel hypersensitivity loss ensuring protected images cannot be matched even when the query image itself is protected, closing a critical gap in prior methods
- Achieves at least 2x retrieval disruption over state-of-the-art methods across a wide range of black-box FR models with superior visual coherence in video settings
🛡️ Threat Analysis
Protego crafts adversarial perturbations applied to facial images at inference time to cause face recognition models to fail matching — a classic input manipulation/evasion attack applied as a user-side privacy defense. The novel hypersensitivity loss and 3D mask approach are contributions to adversarial perturbation methodology.