ML Security PapersAI-Driven Cybersecurity Threats: A Survey of Emerging Risks and Defensive Strategies
Sai Teja Erukude 1, Viswa Chaitanya Marella 2, Suhasnadh Reddy Veluru 2
Published on arXiv
2601.03304
Output Integrity Attack
OWASP ML Top 10 — ML09
Input Manipulation Attack
OWASP ML Top 10 — ML01
Data Poisoning Attack
OWASP ML Top 10 — ML02
Key Finding
Current deepfake detection benchmarks show AUC drops of 45–50% in real-world conditions, highlighting critical robustness gaps across audio, video, and image modalities.
Artificial Intelligence's dual-use nature is revolutionizing the cybersecurity landscape, introducing new threats across four main categories: deepfakes and synthetic media, adversarial AI attacks, automated malware, and AI-powered social engineering. This paper aims to analyze emerging risks, attack mechanisms, and defense shortcomings related to AI in cybersecurity. We introduce a comparative taxonomy connecting AI capabilities with threat modalities and defenses, review over 70 academic and industry references, and identify impactful opportunities for research, such as hybrid detection pipelines and benchmarking frameworks. The paper is structured thematically by threat type, with each section addressing technical context, real-world incidents, legal frameworks, and countermeasures. Our findings emphasize the urgency for explainable, interdisciplinary, and regulatory-compliant AI defense systems to maintain trust and security in digital ecosystems.
Key Contributions
- Comparative taxonomy mapping AI capabilities to four threat modalities (deepfakes, adversarial attacks, automated malware, AI-powered social engineering) with corresponding defensive strategies
- Review of over 70 academic and industry references spanning 2017–2025, including real-world incident case studies and legal/regulatory frameworks
- Identification of research gaps including hybrid detection pipelines, multi-modal deepfake detection, and benchmarking frameworks for AI-driven threats
🛡️ Threat Analysis
Dedicated section on adversarial AI attacks covers gradient-based evasion attacks (FGSM, PGD, C&W), toolkits (CleverHans, ART), model evasion in autonomous vehicles, and defenses including adversarial training, defensive distillation, and certified robustness.
Data poisoning is identified as a distinct threat category in both the introduction and taxonomy — covers manipulation of training data to compromise AI systems and associated defensive strategies.
Major section on deepfakes and synthetic media covers AI-generated content detection, wavelet-based detection methods, benchmark limitations (Deepfake-Eval-2024), and detection challenges — directly output integrity and AI content provenance.