Do AI content detectors work? What the evidence says

By Thomas Lovas Lokoy · Published 16 October 2026 · 7 min read

A man sweeping a metal detector along a beach — Photo: Man metal detector beach · CC0

An AI content detector gives you a percentage and a verdict, and the number looks like a measurement. It is a guess about the statistics of the text, and the independent research on these tools is consistent: they miss a large share of machine-written text, they flag some human writing as machine-written, and the people they flag wrongly are not a random sample. This guide sets out what the published evidence shows, what the tools are actually measuring, and what to do instead if your real question is whether a piece of writing can be trusted.

What an AI content detector actually measures

A detector does not find a hidden signature of authorship. Most of the tools tested in the studies below look at how predictable the text is to a language model. The usual measure is perplexity: roughly, how surprised a model is by each next word. Machine-generated text tends to be low-perplexity, because the model that wrote it chose likely words. Human writing tends to be higher.

The trouble is that low perplexity is not unique to machines. Plain, simple, formulaic prose written by a person can be just as predictable. That one fact explains most of the failures described below.

Researchers group detection methods into a few families. Sadasivan and colleagues, in a paper published in Transactions on Machine Learning Research, stress-tested four: watermark-based detectors (where the model provider deliberately marks its output), neural network classifiers trained on examples, zero-shot methods that use statistics such as perplexity, and retrieval-based detectors that compare a text against a stored database of a model's past outputs. The free and commercial checkers most people paste text into are in the second and third groups.

The broadest independent test: 14 tools, 54 documents

The most useful single study for a publisher is Weber-Wulff and seven co-authors, "Testing of Detection Tools for AI-Generated Text", published in the International Journal for Educational Integrity in 2023. It is useful because the researchers controlled the ground truth. They tested 12 publicly available tools and two commercial systems, Turnitin and PlagiarismCheck, against 54 documents prepared by nine researchers in six categories:

Human-written: about 10,000 characters each, never exposed to the internet.
Machine-translated: human text written in another language and translated into English with DeepL or Google Translate.
AI-generated: two sets of text generated by ChatGPT.
Manually edited: ChatGPT text edited by a person swapping in synonyms and reordering sentence parts.
Machine-paraphrased: ChatGPT text rewritten by the paraphrasing tool Quillbot.

The conclusion, in the authors' words: "the available detection tools are neither accurate nor reliable." The arXiv version of the paper reports that every tool scored below 80% accuracy and only five scored above 70%. The tools leaned towards calling text human, and the authors estimate that about 20% of plainly AI-generated texts would be attributed to people.

The spread between tools matters as much as the average. In that version of the paper, the probability of a false positive (a human text flagged as AI) ranged from 0% for Turnitin to 50% for GPT Zero, and the probability of a false negative ranged from 8% for GPT Zero to 100% for Content at Scale. A tool that rarely misses AI text was, in this test, the one most likely to accuse a human. The study used the ChatGPT model of 13 February 2023, so the specific scores are dated; the pattern of trade-offs is the finding that has held.

The paper also notes that some vendors urge caution themselves, citing OpenAI's own January 2023 explanation that there are several ways to deceive its tool.

Who gets falsely flagged

The second study every editor should know is Liang, Yuksekgonul, Mao, Wu and Zou, "GPT detectors are biased against non-native English writers", published in the journal Patterns. Its abstract puts the finding plainly: the detectors "consistently misclassify non-native English writing samples as AI-generated".

The researchers ran seven widely used detectors over 91 human-written TOEFL essays from a Chinese educational forum and 88 US eighth-grade essays from the Hewlett Foundation's ASAP dataset. The detectors were near-perfect on the eighth-grade essays. On the TOEFL essays, written by people learning English, the average false positive rate was 61.22%. All seven detectors agreed on 18 of the 91 essays (19.78%) being AI-written, and 89 of the 91 (97.80%) were flagged by at least one detector.

The authors then tested the perplexity explanation directly:

They asked ChatGPT to "Enhance the word choices to sound more like that of a native speaker" in the TOEFL essays. The average false positive rate fell from 61.22% to 11.77%.
They asked ChatGPT to simplify the word choices in the eighth-grade essays as if written by a non-native speaker. The misclassification rate rose from an average of 5.19% to 56.65%.

In other words, the detectors were largely measuring vocabulary range. The same mechanism puts anyone who writes in plain, simple English at risk: second-language writers, people writing to a strict house style, and writers deliberately aiming for accessibility. Weber-Wulff's team found a related effect: machine translation of a writer's own text can raise false positives, putting second-language students "at risk of being falsely accused" when they translate their own work.

How easily detection is evaded

The same two studies show the opposite failure. Liang's team generated 31 college admission essays with ChatGPT-3.5. Detectors caught them, until a second prompt asked the model to "Elevate the provided text by employing literary language". Detection fell from 100% to 13%. For 145 generated scientific abstracts, detection of up to 68% fell to up to 28% after a similar rewrite prompt.

Weber-Wulff's team found that light obfuscation was enough: after manual editing, about half of the AI-generated texts went undetected, and machine paraphrasing did worse still for the detectors.

Krishna and colleagues, in a NeurIPS 2023 paper, built a dedicated paraphrasing model called DIPPER. Their abstract reports that it "drops detection accuracy of DetectGPT from 70.3% to 4.6% (at a constant false positive rate of 1%)".

Can reliable detection work in principle?

Not in the form most people use. Sadasivan's paper found that recursive paraphrasing (rewriting the output, then rewriting the rewrite) substantially lowered detection rates across all four detector families, often with only a slight loss of text quality. It also demonstrated spoofing, where human text is made to look AI-generated, and set out a theoretical argument linking the best achievable detector's performance to how different machine text and human text are as distributions: the closer they get, the less any detector can do.

There is one partial exception. Krishna's team showed that a retrieval defence, in which the model provider stores everything its model generates and checks a suspect text against that store, could detect 80% to 97% of paraphrased generations while flagging only 1% of human-written text. That works only if the provider builds and runs it, and only for that provider's model. It is not something a checker website can offer you about text from an unknown source.

Does Google use an AI content detector against your site?

Google's published position makes the question mostly irrelevant for search. Its February 2023 guidance on AI-generated content says: "Our focus on the quality of content, rather than how content is produced, is a useful guide". The same post draws the line at purpose: "Using automation—including AI—to generate content with the primary purpose of manipulating ranking in search results is a violation of our spam policies."

Its current page on using generative AI content adds the practical requirement that matters to anyone publishing: "It is critical to manually factcheck and review all AI-generated content for accuracy and trustworthiness before publishing." Running your draft through a detector and editing until the score drops does none of that. The guide to AI-generated content and Google Search covers the rest of that guidance.

What to do instead

If you are an editor, a client or a teacher, the evidence points to a different set of checks:

Never treat a detector score as proof. Weber-Wulff's team concluded that detector reports "cannot be used as the only basis for reporting students for cheating", because the tools give no evidence a person could contest. The same applies to rejecting a freelancer's work.
Ask for the process, not the verdict. Drafts, notes, version history and sources say far more about how a piece was made than any statistic about its words.
Check what the text claims. Open every source, find every number, verify every name. A mechanical routine for this is in how to fact-check an AI-written article. A wrong figure is a problem whoever typed it.
Judge the writing on its job. Does it answer the reader's question, add something a competitor's page does not, and sound like the publication it appears in? If the voice is the issue, the fix is in the brief; see AI writing in your own voice.
If you write plainly, keep your evidence. Second-language writers and anyone writing in simple English are the people most likely to be wrongly flagged. Drafts and dated files are the defence.

The honest summary of the research is short. Detectors can sometimes spot unedited output from a known model. They cannot reliably tell you who wrote a text that anyone has bothered to edit, and they are least reliable for exactly the writers who most need to be treated fairly.