Melody or Machine: Detecting Synthetic Music with Dual-Stream Contrastive Learning

The rapid evolution of end-to-end AI music generation poses an escalating threat to artistic authenticity and copyright, demanding detection methods that can keep pace. While foundational, existing models like SpecTTTra falter when faced with the diverse and rapidly advancing ecosystem of new generators, exhibiting significant performance drops on out-of-distribution (OOD) content. This generalization failure highlights a critical gap: the need for more challenging benchmarks and more robust detection architectures. To address this, we first introduce Melody or Machine (MoM), a new large-scale benchmark of over 130,000 songs (6,665 hours). MoM is the most diverse dataset to date, built with a mix of open and closed-source models and a curated OOD test set designed specifically to foster the development of truly generalizable detectors. Alongside this benchmark, we introduce CLAM, a novel dual-stream detection architecture. We hypothesize that subtle, machine-induced inconsistencies between vocal and instrumental elements, often imperceptible in a mixed signal, offer a powerful tell-tale sign of synthesis. CLAM is designed to test this hypothesis by employing two distinct pre-trained audio encoders (MERT and Wave2Vec2) to create parallel representations of the audio. These representations are fused by a learnable cross-aggregation module that models their inter-dependencies. The model is trained with a dual-loss objective: a standard binary cross-entropy loss for classification, complemented by a contrastive triplet loss which trains the model to distinguish between coherent and artificially mismatched stream pairings, enhancing its sensitivity to synthetic artifacts without presuming a simple feature alignment. CLAM establishes a new state-of-the-art in synthetic music forensics. It achieves an F1 score of 0.925 on our challenging MoM benchmark.

Key Contributions

• MoM: a large-scale benchmark of 130,435 songs (6,665 hours) spanning 9 generative models with a dedicated out-of-distribution test set to measure real-world generalization
• CLAM: a dual-stream detection architecture using MERT and Wave2Vec2 encoders fused via a cross-aggregation module, trained with BCE + contrastive triplet loss to exploit vocal-instrumental inconsistencies as a forensic signal
• State-of-the-art results: F1 of 0.925 on MoM (vs. prior SOTA 0.869) and 0.993 on SONICS benchmark

🛡️ Threat Analysis

Details

Similar Papers