ML Security PapersAre DeepFakes Realistic Enough? Exploring Semantic Mismatch as a Novel Challenge
Sharayu Nilesh Deshmukh 1,2, Kailash A. Hambarde 1,2, Joana C. Costa 1,2, Hugo Proença 1,2, Tiago Roxo 1,2
Published on arXiv
2604.28022
Output Integrity Attack
OWASP ML Top 10 — ML09
Key Finding
Demonstrates that state-of-the-art four-class deepfake detectors fail to detect semantic mismatches between authentic modalities, and that semantic reinforcement with ImageBind embeddings enables robust five-class detection
RARV-SMM
Novel technique introduced
Current DeepFake detection scenarios are mostly binary, yet data manipulation can vary across audio, video, or both, whose variability is not captured in binary settings. Four-class audio-visual formulations address this by discriminating manipulation type, but introduce a unresolved problem: models may rely solely on data source integrity to detect DeepFakes without evaluating their semantic consistency. If the DeepFake origin is not in the data source but in its content, can semantic mismatch be assessed by the state-of-the-art? This paper proposes a new evaluation setup, extending the four-class formulation by explicitly modeling semantic-level inconsistency between authentic modalities with the introduction a new class: Real Audio-Real Video with Semantic Mismatch (RARV-SMM). We assess the robustness of state-of-the-art models in this new realistic DeepFake setting, using the FakeAVCeleb dataset, highlighting the limitations of existing approaches when faced with semantic mismatch data. We further introduce three RARV-SMM variants that expose distinct architectural vulnerabilities as audio-visual divergence increases. We also propose a semantic reinforcement strategy that incorporates the semantic mismatch class and ImageBind embeddings to improve DeepFake detection in both our proposed and state-of-the-art settings, on FakeAVCeleb and LAV-DF, paving the way to more realistic DeepFake detectors. The source code and data are available at https://github.com/.
Key Contributions
- Introduces RARV-SMM (Real Audio-Real Video with Semantic Mismatch) as a fifth class in deepfake detection, addressing semantic-level inconsistencies that binary and four-class formulations miss
- Proposes three RARV-SMM variants exposing distinct architectural vulnerabilities as audio-visual semantic divergence increases
- Develops a semantic reinforcement strategy using ImageBind embeddings to improve deepfake detection across both proposed and state-of-the-art settings on FakeAVCeleb and LAV-DF datasets
🛡️ Threat Analysis
Paper focuses on detecting AI-generated/manipulated audio-visual content (deepfakes) and proposes a new detection approach that evaluates output integrity through semantic consistency analysis. The core contribution is a defense method for verifying the authenticity of audio-visual outputs by detecting semantic mismatches between real modalities.