Diffusion-Driven Deceptive Patches: Adversarial Manipulation and Forensic Detection in Facial Identity Verification

This work presents an end-to-end pipeline for generating, refining, and evaluating adversarial patches to compromise facial biometric systems, with applications in forensic analysis and security testing. We utilize FGSM to generate adversarial noise targeting an identity classifier and employ a diffusion model with reverse diffusion to enhance imperceptibility through Gaussian smoothing and adaptive brightness correction, thereby facilitating synthetic adversarial patch evasion. The refined patch is applied to facial images to test its ability to evade recognition systems while maintaining natural visual characteristics. A Vision Transformer (ViT)-GPT2 model generates captions to provide a semantic description of a person's identity for adversarial images, supporting forensic interpretation and documentation for identity evasion and recognition attacks. The pipeline evaluates changes in identity classification, captioning results, and vulnerabilities in facial identity verification and expression recognition under adversarial conditions. We further demonstrate effective detection and analysis of adversarial patches and adversarial samples using perceptual hashing and segmentation, achieving an SSIM of 0.95.

Key Contributions

• End-to-end pipeline combining FGSM adversarial noise generation with diffusion model reverse diffusion, Gaussian smoothing, and adaptive brightness correction to produce imperceptible adversarial patches against facial identity classifiers
• ViT-GPT2 captioning module that semantically describes adversarial facial images for forensic documentation of identity evasion attacks
• Adversarial patch detection and forensic analysis using perceptual hashing and segmentation, achieving 0.95 SSIM between original and reconstructed adversarial images

🛡️ Threat Analysis

Details

Similar Papers