Skip to content

A survey and reflection on the latest research breakthroughs in LLM-generated Text detection, including data, detectors, metrics, current issues and future directions.

Notifications You must be signed in to change notification settings

JenniferYingyiWu2020/LLM-generated-Text-Detection

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

32 Commits
 
 
 
 
 
 
 
 

Repository files navigation

Awesome LLM-generated Text Detection

Logo

Awesome License-MIT PRs-Welcome last-commit

The powerful ability of large language models (LLMs) to understand, follow, and generate complex languages has enabled LLM-generated texts to flood many areas of our daily lives at an incredible rate, with potentially negative impacts and risks on society and academia. As LLMs continue to expand, how can we detect LLM-generated texts to help minimize the threat posed by the misuse of LLMs?

cover

¹ Junchao Wu, ¹ Shu Yang, ¹ Runzhe Zhan, ¹ ² Yulin Yuan, ¹ Derek Fai Wong, ¹ Lidia Sam Chao

¹ University of Macau, ² Peking University

📢 News

🔍 Table of Contents

📃 Papers

Overview

A survey and reflection on the latest research breakthroughs in LLM-generated Text detection, including data, detectors, metrics, current issues and future directions. Please refer to our article/paper for more details.

Datasets

Benchmarks

Benchmarks / Datasets Use Human LLMs
HC3 train 58k 26k
HC3-Chinese train 22k 17k
CHEAT train 15k 35k
GROVER Dataset train valid test 5k 2k 8k 5k 1k 4k
TweepFake train 12k 12k
GPT-2 Output Dataset train 250k 250k
TuringBench train 10k 190k
MGTBench train test 2k 563 13k 3k
ArguGPT train valid test 3k 350 350 3k 350 350
DeepfakeText-Detect-Dataset train valid test 95k 29k 29k 236k, 29k 28k
M4 train valid test 122k 500 500 122k 500 500
GPABenchmark train 600k 600k
Scientific-articles Benchmark train test 8k 4k 8k 4k

Potential Datasets

Tasks Datasets
Questions Answering PubMedQA, Children book corpus (CBT), ELI5, TruthfulQA, NarrativeQA
Scientific writing Peer Read, arXiv, TOEFL11
Story generation WritingPrompts
News Article writing XSum
Web Text Wiki40b, WebText, Avax tweets dataset, Climate Change Tweets Ids
Opinion statements r/ChangeMyView (CMV) Reddit subcommunity, Yelp , IMDB Dataset
Comprehension and Reasoning SciGen, ROCStories Corpora, HellaSwag, SQuAD

Detectors

Detector

Watermark Technology

Paper Link
A watermark for large language models. Static Badge Static Badge
On the Reliability of Watermarks for Large Language Models Static Badge Static Badge
A Private Watermark for Large Language Models Static Badge Static Badge
Distillation-Resistant Watermarking for Model Protection in NLP Static Badge Static Badge
Watermarking Pre-trained Language Models with Backdooring Static Badge

Zero-shot Methods

Paper Link
DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature Static Badge Static Badge
Fast-DetectGPT: Efficient Zero-Shot Detection of Machine-Generated Text via Conditional Probability Curvature Static Badge Static Badge
Efficient Detection of LLM-generated Texts with a Bayesian Surrogate Model Static Badge
DetectLLM: Leveraging Log Rank Information for Zero-Shot Detection of Machine-Generated Text Static Badge Static Badge
GLTR: Statistical Detection and Visualization of Generated Text Static Badge Static Badge
HowkGPT: Investigating the Detection of ChatGPT-generated University Student Homework through Context-Aware Perplexity Analysis Static Badge
Intrinsic Dimension Estimation for Robust Detection of AI-Generated Texts Static Badge

Fine-tuning LMs Methods

Paper Link
How Close is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and Detection Static Badge Static Badge
Multiscale Positive-Unlabeled Detection of AI-Generated Texts Static Badge Static Badge
Real or fake? Learning to discriminate machine from human generated text Static Badge
Automatic Detection of Generated Text is Easiest when Humans are Fooled Static Badge
Stylometric Detection of AI-Generated Text in Twitter Timelines Static Badge
TweepFake: about Detecting Deepfake Tweets Static Badge Static Badge
Towards a Robust Detection of Language Model Generated Text: Is ChatGPT that Easy to Detect? Static Badge
Deepfake Text Detection in the Wild Static Badge Static Badge
ArguGPT: evaluating, understanding and identifying argumentative essays generated by GPT models Static Badge Static Badge
Check Me If You Can: Detecting ChatGPT-Generated Academic Writing using CheckGPT Static Badge
GPT-Sentinel: Distinguishing Human and ChatGPT Generated Content Static Badge
Neural Deepfake Detection with Factual Structure of Text Static Badge
ConDA: Contrastive Domain Adaptation for AI-generated Text Detection Static Badge Static Badge

Adversarial Learning Methods

Paper Link
RADAR: Robust AI-Text Detection via Adversarial Learning Static Badge
OUTFOX: LLM-generated Essay Detection through In-context Learning with Adversarially Generated Examples Static Badge Static Badge
Red Teaming Language Model Detectors with Language Models Static Badge Static Badge
Is ChatGPT Involved in Texts? Measure the Polish Ratio to Detect ChatGPT-Generated Text Static Badge

LLMs as Detector

Paper Link
Fighting fire with fire: Can chatgpt detect ai-generated text? Static Badge Static Badge
OUTFOX: LLM-generated Essay Detection through In-context Learning with Adversarially Generated Examples Static Badge Static Badge
GPT Paternity Test: GPT Generated Text Detection with GPT Genetic Inheritance Static Badge

Related Works

Other Surveys

Paper Link
Automatic Detection of Machine Generated Text: A Critical Survey Static Badge
The Science of Detecting LLM-Generated Texts Static Badge
Machine Generated Text: A Comprehensive Survey of Threat Models and Detection Methods Static Badge
Computer-Generated Text Detection Using Machine Learning: A Systematic Review Static Badge
Attribution and Obfuscation of Neural Text Authorship: A Data Mining Perspective Static Badge

🚩 Citation

If our research helps you, please kindly cite our paper.

@article{wu2023survey,
      title={A Survey on LLM-gernerated Text Detection: Necessity, Methods, and Future Directions}, 
      author={Junchao Wu and Shu Yang and Runzhe Zhan and Yulin Yuan and Derek F. Wong and Lidia S. Chao},
      journal      = {CoRR},
      volume       = {abs/2310.14724},
      year         = {2023},
      url          = {https://arxiv.org/abs/2310.14724},
      eprinttype   = {arXiv},
      eprint       = {2310.14724},

Contributing

Contributions are welcome! If you have any ideas, suggestions, or bug reports, please open an issue or submit a pull request. We appreciate your contributions to making LLM-generated Text Detection work even better.

About

A survey and reflection on the latest research breakthroughs in LLM-generated Text detection, including data, detectors, metrics, current issues and future directions.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published