ML Security PapersArtifactLens: Hundreds of Labels Are Enough for Artifact Detection with VLMs
James Burgess 1, Rameen Abdal 2, Dan Stoddart 2, Sergey Tulyakov 2, Serena Yeung-Levy 1, Kuan-Chieh Jackson Wang 2
Published on arXiv
2602.09475
Output Integrity Attack
OWASP ML Top 10 — ML09
Key Finding
Achieves state-of-the-art on five human artifact benchmarks using orders of magnitude less labeled data (hundreds vs. tens of thousands of examples) compared to fine-tuned VLM baselines
ArtifactLens
Novel technique introduced
Modern image generators produce strikingly realistic images, where only artifacts like distorted hands or warped objects reveal their synthetic origin. Detecting these artifacts is essential: without detection, we cannot benchmark generators or train reward models to improve them. Current detectors fine-tune VLMs on tens of thousands of labeled images, but this is expensive to repeat whenever generators evolve or new artifact types emerge. We show that pretrained VLMs already encode the knowledge needed to detect artifacts - with the right scaffolding, this capability can be unlocked using only a few hundred labeled examples per artifact category. Our system, ArtifactLens, achieves state-of-the-art on five human artifact benchmarks (the first evaluation across multiple datasets) while requiring orders of magnitude less labeled data. The scaffolding consists of a multi-component architecture with in-context learning and text instruction optimization, with novel improvements to each. Our methods generalize to other artifact types - object morphology, animal anatomy, and entity interactions - and to the distinct task of AIGC detection.
Key Contributions
- ArtifactLens: a multi-component scaffold around frozen VLMs using specialist sub-models, in-context learning, and black-box text instruction optimization, requiring only hundreds of labeled examples per artifact category
- First evaluation of AI-generated image artifact detectors across five human artifact benchmarks simultaneously, achieving state-of-the-art results
- Demonstrated generalization to object morphology, animal anatomy, entity interactions, and AIGC detection tasks
🛡️ Threat Analysis
The paper's primary contribution is an AI-generated content (AIGC) detection system — detecting synthetic image artifacts (distorted hands, warped objects) produced by generative models. This is directly about verifying the authenticity and origin of visual content, a core ML09 concern. The paper explicitly evaluates on the 'distinct task of AIGC detection' alongside artifact detection.