ML Security PapersA Lightweight Approach to Detection of AI-Generated Texts Using Stylometric Features
Sergey K. Aityan , William Claster , Karthik Sai Emani , Sohni Rais , Thy Tran
Published on arXiv
2511.21744
Output Integrity Attack
OWASP ML Top 10 — ML09
Key Finding
NEULIF CNN achieves 97% accuracy (0.95 F1, 99.5% ROC-AUC) with a 25 MB model runnable on CPU, matching transformer-based ensemble accuracy at a fraction of the compute cost.
NEULIF
Novel technique introduced
A growing number of AI-generated texts raise serious concerns. Most existing approaches to AI-generated text detection rely on fine-tuning large transformer models or building ensembles, which are computationally expensive and often provide limited generalization across domains. Existing lightweight alternatives achieved significantly lower accuracy on large datasets. We introduce NEULIF, a lightweight approach that achieves best performance in the lightweight detector class, that does not require extensive computational power and provides high detection accuracy. In our approach, a text is first decomposed into stylometric and readability features which are then used for classification by a compact Convolutional Neural Network (CNN) or Random Forest (RF). Evaluated and tested on the Kaggle AI vs. Human corpus, our models achieve 97% accuracy (~ 0.95 F1) for CNN and 95% accuracy (~ 0.94 F1) for the Random Forest, demonstrating high precision and recall, with ROC-AUC scores of 99.5% and 95%, respectively. The CNN (~ 25 MB) and Random Forest (~ 10.6 MB) models are orders of magnitude smaller than transformer-based ensembles and can be run efficiently on standard CPU devices, without sacrificing accuracy. This study also highlights the potential of such models for broader applications across languages, domains, and streaming contexts, showing that simplicity, when guided by structural insights, can rival complexity in AI-generated content detection.
Key Contributions
- NEULIF: a lightweight AI-generated text detector decomposing text into stylometric and readability features for classification by a CNN or Random Forest
- CNN (~25 MB) achieving 97% accuracy / 0.95 F1 / 99.5% ROC-AUC and RF (~10.6 MB) achieving 95% accuracy on Kaggle AI vs. Human corpus — orders of magnitude smaller than transformer ensembles
- Demonstrates that CPU-deployable, streaming-compatible detectors can rival transformer-based ensembles using structural/stylistic signals alone
🛡️ Threat Analysis
Proposes a novel method to detect AI-generated text — directly addresses output integrity and content authenticity, a core ML09 concern.