Learning to Forget with Information Divergence Reweighted Objectives for Noisy Labels

We introduce ANTIDOTE, a new class of objectives for learning under noisy labels which are defined in terms of a relaxation over an information-divergence neighborhood. Using convex duality, we provide a reformulation as an adversarial training method that has similar computational cost to training with standard cross-entropy loss. We show that our approach adaptively reduces the influence of the samples with noisy labels during learning, exhibiting a behavior that is analogous to forgetting those samples. ANTIDOTE is effective in practical environments where label noise is inherent in the training data or where an adversary can alter the training labels. Extensive empirical evaluations on different levels of symmetric, asymmetric, human annotation, and real-world label noise show that ANTIDOTE outperforms leading comparable losses in the field and enjoys a time complexity that is very close to that of the standard cross entropy loss.

Key Contributions

• Introduces ANTIDOTE, a theoretically grounded class of loss objectives based on f-divergence neighborhood relaxation that adaptively down-weights noisy/poisoned labels during training.
• Provides a convex duality reformulation enabling efficient adversarial training with computational cost near that of standard cross-entropy loss.
• Proves that true labels are the unique solution to the relaxed problem even in the presence of label noise, and demonstrates state-of-the-art performance across symmetric, asymmetric, human-annotation, and real-world label noise settings.

🛡️ Threat Analysis

Details

Similar Papers