benchmark arXiv Dec 15, 2025 · Dec 2025
Ali Al Sahili, Ali Chehab, Razane Tajeddine · American University of Beirut
Benchmarks MIA techniques integrated into LLM training data extraction pipelines versus standalone MIA evaluation settings
Membership Inference Attack Model Inversion Attack Sensitive Information Disclosure nlp
Large Language Models (LLMs) are prone to memorizing training data, which poses serious privacy risks. Two of the most prominent concerns are training data extraction and Membership Inference Attacks (MIAs). Prior research has shown that these threats are interconnected: adversaries can extract training data from an LLM by querying the model to generate a large volume of text and subsequently applying MIAs to verify whether a particular data point was included in the training set. In this study, we integrate multiple MIA techniques into the data extraction pipeline to systematically benchmark their effectiveness. We then compare their performance in this integrated setting against results from conventional MIA benchmarks, allowing us to evaluate their practical utility in real-world extraction scenarios.
llm transformer American University of Beirut
ML Security Papers