LoRA Patching: Exposing the Fragility of Proactive Defenses against Deepfakes

Deepfakes pose significant societal risks, motivating the development of proactive defenses that embed adversarial perturbations in facial images to prevent manipulation. However, in this paper, we show that these preemptive defenses often lack robustness and reliability. We propose a novel approach, Low-Rank Adaptation (LoRA) patching, which injects a plug-and-play LoRA patch into Deepfake generators to bypass state-of-the-art defenses. A learnable gating mechanism adaptively controls the effect of the LoRA patch and prevents gradient explosions during fine-tuning. We also introduce a Multi-Modal Feature Alignment (MMFA) loss, encouraging the features of adversarial outputs to align with those of the desired outputs at the semantic level. Beyond bypassing, we present defensive LoRA patching, embedding visible warnings in the outputs as a complementary solution to mitigate this newly identified security vulnerability. With only 1,000 facial examples and a single epoch of fine-tuning, LoRA patching successfully defeats multiple proactive defenses. These results reveal a critical weakness in current paradigms and underscore the need for more robust Deepfake defense strategies. Our code is available at https://github.com/ZOMIN28/LoRA-Patching.

Key Contributions

• LoRA patching: injects plug-and-play LoRA blocks into GAN-based deepfake generators to neutralize adversarial perturbations from proactive defenses
• Learnable gating mechanism to stabilize fine-tuning and Multi-Modal Feature Alignment (MMFA) loss to align adversarial and benign output features
• Defensive LoRA patching variant that embeds visible warning watermarks in deepfake outputs as a countermeasure to the identified vulnerability

🛡️ Threat Analysis

Details

Similar Papers