ML Security PapersRealStats: A Rigorous Real-Only Statistical Framework for Fake Image Detection
Published on arXiv
2601.18900
Output Integrity Attack
OWASP ML Top 10 — ML09
Key Finding
RealStats produces calibrated p-values with formal statistical meaning, improving interpretability over supervised classifiers and maintaining robustness to unseen generative models without requiring training on fake data.
RealStats
Novel technique introduced
As generative models continue to evolve, detecting AI-generated images remains a critical challenge. While effective detection methods exist, they often lack formal interpretability and may rely on implicit assumptions about fake content, potentially limiting robustness to distributional shifts. In this work, we introduce a rigorous, statistically grounded framework for fake image detection that focuses on producing a probability score interpretable with respect to the real-image population. Our method leverages the strengths of multiple existing detectors by combining training-free statistics. We compute p-values over a range of test statistics and aggregate them using classical statistical ensembling to assess alignment with the unified real-image distribution. This framework is generic, flexible, and training-free, making it well-suited for robust fake image detection across diverse and evolving settings.
Key Contributions
- Formulates AI-generated image detection as a formal hypothesis test against the real-image distribution, producing statistically interpretable p-values instead of opaque classifier scores.
- Training-free ensemble framework that aggregates multiple existing detection statistics via classical p-value combining methods (e.g., Fisher, Stouffer) without requiring any fake image data.
- Generic and extensible architecture that improves adaptability to distribution shifts from emerging generators by relying solely on real-image reference distributions.
🛡️ Threat Analysis
Proposes a novel AI-generated image detection framework that produces calibrated p-values to authenticate whether an image comes from the real-image distribution — directly addresses output integrity and content authenticity for synthetic image detection.