ML Security PapersTowards Effective Prompt Stealing Attack against Text-to-Image Diffusion Models
Shiqian Zhao 1, Chong Wang 1, Yiming Li 1, Yihao Huang 2, Wenjie Qu 2, Siew-Kei Lam 1, Yi Xie 3, Kangjie Chen 1, Jie Zhang 4, Tianwei Zhang 1
Published on arXiv
2508.06837
Model Theft
OWASP ML Top 10 — ML05
Sensitive Information Disclosure
OWASP LLM Top 10 — LLM06
Key Finding
Prometheus achieves a 25% attack success rate improvement over PromptStealer against real commercial T2I platforms (Midjourney, DALL·E 3, Leonardo.ai) while resisting extensive potential defenses.
Prometheus
Novel technique introduced
Text-to-Image (T2I) models, represented by DALL$\cdot$E and Midjourney, have gained huge popularity for creating realistic images. The quality of these images relies on the carefully engineered prompts, which have become valuable intellectual property. While skilled prompters showcase their AI-generated art on markets to attract buyers, this business incidentally exposes them to \textit{prompt stealing attacks}. Existing state-of-the-art attack techniques reconstruct the prompts from a fixed set of modifiers (i.e., style descriptions) with model-specific training, which exhibit restricted adaptability and effectiveness to diverse showcases (i.e., target images) and diffusion models. To alleviate these limitations, we propose Prometheus, a training-free, proxy-in-the-loop, search-based prompt-stealing attack, which reverse-engineers the valuable prompts of the showcases by interacting with a local proxy model. It consists of three innovative designs. First, we introduce dynamic modifiers, as a supplement to static modifiers used in prior works. These dynamic modifiers provide more details specific to the showcases, and we exploit NLP analysis to generate them on the fly. Second, we design a contextual matching algorithm to sort both dynamic and static modifiers. This offline process helps reduce the search space of the subsequent step. Third, we interact with a local proxy model to invert the prompts with a greedy search algorithm. Based on the feedback guidance, we refine the prompt to achieve higher fidelity. The evaluation results show that Prometheus successfully extracts prompts from popular platforms like PromptBase and AIFrog against diverse victim models, including Midjourney, Leonardo.ai, and DALL$\cdot$E, with an ASR improvement of 25.0\%. We also validate that Prometheus is resistant to extensive potential defenses, further highlighting its severity in practice.
Key Contributions
- Prometheus: a training-free, proxy-in-the-loop, greedy search-based prompt stealing attack that reverse-engineers T2I prompts without model-specific training
- Dynamic modifier generation via NLP analysis to overcome the OOV limitations of fixed modifier vocabularies used in prior work (PromptStealer)
- Contextual matching algorithm + proxy model feedback loop that achieves 25% ASR improvement over SOTA against Midjourney, DALL·E, and Leonardo.ai
🛡️ Threat Analysis
The paper's primary contribution is IP theft of valuable prompt engineering work — it steals commercially traded prompts (priced up to $15 each on PromptBase/AIFrog) by reconstructing them from showcase images. The paper explicitly frames this as intellectual property theft in the AI ecosystem. While ML05 traditionally covers model weight extraction, the spirit of ML05 (protecting AI-generated intellectual property from theft) is most closely aligned with this attack, which steals the 'learned functionality' of prompt engineers from T2I systems.