DependencyAI: Detecting AI Generated Text through Dependency Parsing

As large language models (LLMs) become increasingly prevalent, reliable methods for detecting AI-generated text are critical for mitigating potential risks. We introduce DependencyAI, a simple and interpretable approach for detecting AI-generated text using only the labels of linguistic dependency relations. Our method achieves competitive performance across monolingual, multi-generator, and multilingual settings. To increase interpretability, we analyze feature importance to reveal syntactic structures that distinguish AI-generated from human-written text. We also observe a systematic overprediction of certain models on unseen domains, suggesting that generator-specific writing styles may affect cross-domain generalization. Overall, our results demonstrate that dependency relations alone provide a robust signal for AI-generated text detection, establishing DependencyAI as a strong linguistically grounded, interpretable, and non-neural network baseline.

Key Contributions

• DependencyAI: an interpretable AI-text detector using TF-IDF n-grams of dependency relation labels (discarding all lexical content) with a LightGBM classifier
• Evaluation across monolingual, multi-generator, and multilingual settings on M4GT-Bench, establishing a competitive non-neural interpretable baseline
• Feature importance analysis identifying which syntactic dependency structures (e.g., nsubj, ROOT bigrams) most distinguish AI-generated from human-written text

🛡️ Threat Analysis

Details

Similar Papers