SoK: Challenges in Tabular Membership Inference Attacks

Membership Inference Attacks (MIAs) are currently a dominant approach for evaluating privacy in machine learning applications. Despite their significance in identifying records belonging to the training dataset, several concerns remain unexplored, particularly with regard to tabular data. In this paper, first, we provide an extensive review and analysis of MIAs considering two main learning paradigms: centralized and federated learning. We extend and refine the taxonomy for both. Second, we demonstrate the efficacy of MIAs in tabular data using several attack strategies, also including defenses. Furthermore, in a federated learning scenario, we consider the threat posed by an outsider adversary, which is often neglected. Third, we demonstrate the high vulnerability of single-outs (records with a unique signature) to MIAs. Lastly, we explore how MIAs transfer across model architectures. Our results point towards a general poor performance of these attacks in tabular data which contrasts with previous state-of-the-art. Notably, even attacks with limited attack performance can still successfully expose a large portion of single-outs. Moreover, our findings suggest that using different surrogate models makes MIAs more effective.

Key Contributions

• Extended and refined taxonomy of Membership Inference Attacks covering both centralized and federated learning paradigms for tabular data
• Empirical evaluation showing MIAs generally underperform on tabular data, contrasting prior state-of-the-art claims, while still exposing a large fraction of single-out records
• Analysis of outsider adversary threat in federated learning and cross-architecture MIA transferability, showing surrogate model diversity improves attack effectiveness

🛡️ Threat Analysis

Details

Similar Papers