ML Security PapersAttackPilot: Autonomous Inference Attacks Against ML Services With LLM-Based Agents
Yixin Wu 1, Rui Wen 2, Chi Cui 1, Michael Backes 1, Yang Zhang 1
Published on arXiv
2511.19536
Membership Inference Attack
OWASP ML Top 10 — ML04
Model Theft
OWASP ML Top 10 — ML05
Model Inversion Attack
OWASP ML Top 10 — ML03
Key Finding
AttackPilot with GPT-4o achieves 100% task completion and membership inference accuracy only 1.0% below a human expert across 20 target ML services at $0.627/run average cost.
AttackPilot
Novel technique introduced
Inference attacks have been widely studied and offer a systematic risk assessment of ML services; however, their implementation and the attack parameters for optimal estimation are challenging for non-experts. The emergence of advanced large language models presents a promising yet largely unexplored opportunity to develop autonomous agents as inference attack experts, helping address this challenge. In this paper, we propose AttackPilot, an autonomous agent capable of independently conducting inference attacks without human intervention. We evaluate it on 20 target services. The evaluation shows that our agent, using GPT-4o, achieves a 100.0% task completion rate and near-expert attack performance, with an average token cost of only $0.627 per run. The agent can also be powered by many other representative LLMs and can adaptively optimize its strategy under service constraints. We further perform trace analysis, demonstrating that design choices, such as a multi-agent framework and task-specific action spaces, effectively mitigate errors such as bad plans, inability to follow instructions, task context loss, and hallucinations. We anticipate that such agents could empower non-expert ML service providers, auditors, or regulators to systematically assess the risks of ML services without requiring deep domain expertise.
Key Contributions
- First autonomous LLM-based multi-agent framework (AttackPilot) that automates all major inference attack types against ML services without human intervention
- Achieves 100% task completion rate and near-expert performance (GPT-4o) at an average cost of $0.627 per run, outperforming MLAgentBench (26.3% completion) by a large margin
- Trace analysis identifying four error classes in baseline agent frameworks and demonstrating how task-specific action spaces and multi-agent design mitigate them
🛡️ Threat Analysis
Data reconstruction and attribute inference attacks — both targeting private training data recovery — are explicitly included in AttackPilot's automated attack suite.
Membership inference is the most prominently evaluated attack type; AttackPilot achieves within 1.0% of human expert MIA accuracy across 20 target services.
Model stealing (model extraction) is explicitly one of the four core inference attack types automated by AttackPilot.