On the Adversarial Robustness of Learning-based Conformal Novelty Detection

This paper studies the adversarial robustness of conformal novelty detection. In particular, we focus on two powerful learning-based frameworks that come with finite-sample false discovery rate (FDR) control: one is AdaDetect (by Marandon et al., 2024) that is based on the positive-unlabeled classifier, and the other is a one-class classifier-based approach (by Bates et al., 2023). While they provide rigorous statistical guarantees under benign conditions, their behavior under adversarial perturbations remains underexplored. We first formulate an oracle attack setup, under the AdaDetect formulation, that quantifies the worst-case degradation of FDR, deriving an upper bound that characterizes the statistical cost of attacks. This idealized formulation directly motivates a practical and effective attack scheme that only requires query access to the output labels of both frameworks. Coupling these formulations with two popular and complementary black-box adversarial algorithms, we systematically evaluate the vulnerability of both frameworks on synthetic and real-world datasets. Our results show that adversarial perturbations can significantly increase the FDR while maintaining high detection power, exposing fundamental limitations of current error-controlled novelty detection methods and motivating the development of more robust alternatives.

Key Contributions

• Oracle attack formulation under AdaDetect that derives an upper bound on worst-case FDR degradation under adversarial perturbations
• Practical surrogate decision-based attack requiring only query access to output labels of both AdaDetect and Bates et al. frameworks
• Systematic empirical evaluation using HopSkipJump and Boundary Attack demonstrating that adversarial perturbations significantly inflate FDR while preserving detection power in conformal novelty detection

🛡️ Threat Analysis

Details

Similar Papers