ML Security PapersSOGPTSpotter: Detecting ChatGPT-Generated Answers on Stack Overflow
Suyu Ma 1, Chunyang Chen 2, Hourieh Khalajzadeh 3, John Grundy 4
Published on arXiv
2602.04185
Output Integrity Attack
OWASP ML Top 10 — ML09
Key Finding
SOGPTSpotter outperforms all baselines including GPTZero and DetectGPT in detecting ChatGPT-generated Stack Overflow answers, with robustness to adversarial attacks and cross-domain generalization confirmed in ablation studies
SOGPTSpotter
Novel technique introduced
Stack Overflow is a popular Q&A platform where users ask technical questions and receive answers from a community of experts. Recently, there has been a significant increase in the number of answers generated by ChatGPT, which can lead to incorrect and unreliable information being posted on the site. While Stack Overflow has banned such AI-generated content, detecting whether a post is ChatGPT-generated remains a challenging task. We introduce a novel approach, SOGPTSpotter, that employs Siamese Neural Networks, leveraging the BigBird model and the Triplet loss, to detect ChatGPT-generated answers on Stack Overflow. We use triplets of human answers, reference answers, and ChatGPT answers. Our empirical evaluation reveals that our approach outperforms well-established baselines like GPTZero, DetectGPT, GLTR, BERT, RoBERTa, and GPT-2 in identifying ChatGPT-synthesized Stack Overflow responses. We also conducted an ablation study to show the effectiveness of our model. Additional experiments were conducted to assess various factors, including the impact of text length, the model's robustness against adversarial attacks, and its generalization capabilities across different domains and large language models. We also conducted a real-world case study on Stack Overflow. Using our tool's recommendations, Stack Overflow moderators were able to identify and take down ChatGPT-suspected generated answers, demonstrating the practical applicability and effectiveness of our approach.
Key Contributions
- SOGPTSpotter: Siamese Neural Network using BigBird encoder and Triplet loss trained on (human answer, reference answer, ChatGPT answer) triplets for AI-text detection
- Empirical evaluation outperforming GPTZero, DetectGPT, GLTR, BERT, RoBERTa, and GPT-2 on Stack Overflow data
- Real-world deployment with Stack Overflow moderators who used the tool to identify and remove ChatGPT-generated answers
🛡️ Threat Analysis
Core contribution is a novel AI-generated content detection architecture (SOGPTSpotter) that detects LLM-produced text in a Q&A platform — directly addressing output integrity and content provenance through a new forensic detection method.