ML Security PapersAVFakeBench: A Comprehensive Audio-Video Forgery Detection Benchmark for AV-LMMs
Shuhan Xia 1, Peipei Li 1, Xuannan Liu 1, Dongsen Zhang 1, Xinyu Guo 2, Zekun Li 1
Published on arXiv
2511.21251
Output Integrity Attack
OWASP ML Top 10 — ML09
Key Finding
Evaluation of 11 AV-LMMs reveals promising detection potential but notable weaknesses in fine-grained forgery perception and reasoning tasks compared to specialized detection methods.
AVFakeBench
Novel technique introduced
The threat of Audio-Video (AV) forgery is rapidly evolving beyond human-centric deepfakes to include more diverse manipulations across complex natural scenes. However, existing benchmarks are still confined to DeepFake-based forgeries and single-granularity annotations, thus failing to capture the diversity and complexity of real-world forgery scenarios. To address this, we introduce AVFakeBench, the first comprehensive audio-video forgery detection benchmark that spans rich forgery semantics across both human subject and general subject. AVFakeBench comprises 12K carefully curated audio-video questions, covering seven forgery types and four levels of annotations. To ensure high-quality and diverse forgeries, we propose a multi-stage hybrid forgery framework that integrates proprietary models for task planning with expert generative models for precise manipulation. The benchmark establishes a multi-task evaluation framework covering binary judgment, forgery types classification, forgery detail selection, and explanatory reasoning. We evaluate 11 Audio-Video Large Language Models (AV-LMMs) and 2 prevalent detection methods on AVFakeBench, demonstrating the potential of AV-LMMs as emerging forgery detectors while revealing their notable weaknesses in fine-grained perception and reasoning.
Key Contributions
- AVFakeBench: a 12K audio-video question benchmark spanning 7 forgery types (human and general subjects) with 4 annotation granularity levels
- Multi-stage hybrid forgery framework integrating proprietary planning models with expert generative models for diverse, high-quality AV forgeries
- Multi-task evaluation framework (binary judgment, forgery type classification, detail selection, explanatory reasoning) revealing fine-grained perception weaknesses in 11 AV-LMMs
🛡️ Threat Analysis
The benchmark directly targets AI-generated content detection — specifically audio-video forgeries and deepfakes. Creating evaluation frameworks, datasets, and metrics for detecting forged/synthetic multimodal content is a core ML09 contribution (output integrity and AI-generated content detection).