Two very different people type ai detector into a search box in the same hour. One is a marketing manager who has published forty pages drafted with help from a language model and has been told that Google will punish the site for it. The other has been accused of cheating because a tool returned a number and somebody in authority believed it.

Both arrive at a category of software that sells confidence, and both deserve the uncomfortable version of the answer. The published evidence on these tools is consistent and it is not flattering. They are wrong often enough that a score should never decide anything about a person, they collapse on text a human has edited, and their errors are not spread evenly across writers.

The search engine question is a separate question with a separate answer, and that answer has nothing to do with detection.

Are these tools accurate, and does Google penalise AI content? Peer reviewed testing put every tool below 80% accuracy, falling to 26% on AI text that had been paraphrased, and a separate study found detectors labelled non native English essays as AI written 61.3% of the time. Google does not penalise content for being AI generated. It penalises scaled content abuse, which is about purpose and value, not authorship.

How an AI Detector Actually Works

Almost everything sold in this category belongs to one of two families, and the distinction matters because they fail in different ways.

Perplexity and Burstiness

The older family measures statistical properties of the text itself. Perplexity describes how surprised a language model is by each word given the words before it, so prose a model finds highly predictable scores low. Burstiness describes the variation in that predictability from sentence to sentence, on the theory that human writing swings between the obvious and the odd while generated text sits in a narrower band.

A tool built this way is not identifying a machine. It is measuring how conventional the prose is and then asserting that conventional prose is machine prose. That inference is the entire product, and it is where every failure mode comes from.

Classifiers Trained on Generated Text

The newer family trains a model on paired corpora of human and generated text and learns whatever separates them. It performs better than raw perplexity on the distribution it was trained on, and it inherits a harder problem: that distribution shifts every time a model is updated or a prompt changes register.

Neither family has access to any fact about how the text was produced. Both are guessing about style from the finished artefact, which is precisely why changing the style defeats them. A third approach, watermarking, works differently and is covered further down, because it is the only method that starts from evidence rather than inference.

What the Research Says About AI Detector Accuracy

The Fourteen Tool Benchmark

The most cited controlled test is Testing of detection tools for AI-generated text, published in the International Journal for Educational Integrity in 2023. Weber-Wulff and colleagues put 54 test documents through 14 tools, twelve free and two commercial ones (Turnitin and PlagiarismCheck), for 756 individual tests.

The result is blunt. Every tool scored below 80% accuracy and only five cleared 70%. Accuracy on genuinely human written documents was 96%, which reads well until you see the rest. On unmodified AI text it was 74%. On AI text that a human had lightly edited it was 42%. On AI text run through a paraphrasing tool it was 26%.

The paper records that its findings “substantially differ from what some detection tools for AI-generated text claim”, and its own conclusion is quoted far less often than it should be: the tools “are neither accurate nor reliable”, and “the systems we tested should not be used in academic settings”.

A 2026 Test of the Commercial Leaders

Three years later, a February 2026 study in the same journal tested Turnitin and Originality against a balanced set of 192 texts: authentic student writing produced before generative AI was widely available, professional human writing, AI output, and hybrid documents built from roughly half of each.

Originality reached a macro-average accuracy of 0.69 and Turnitin 0.61, and both produced macro-average F1 scores below 0.55. On the hybrid texts, which describe how most people now actually write, Originality showed near zero recall. It identified almost none of them.

The domain gap is the finding that should trouble anyone publishing technical content. Turnitin’s accuracy fell from 0.86 on humanities texts to 0.51 on scientific ones. Originality fell from 0.96 to 0.58. Both differences were statistically significant. At 0.51 on scientific writing, a detector is doing something close to flipping a coin.

The One Vendor That Published Its Own Failure

OpenAI shipped a classifier in January 2023 and withdrew it. Its own announcement page now carries the note that “as of July 20, 2023, the AI classifier is no longer available due to its low rate of accuracy”, and the original text still records the numbers: it “correctly identifies 26% of AI-written text (true positives) as ’likely AI-written,’ while incorrectly labeling human-written text as AI-written 9% of the time”. The page also warned that it “should not be used as a primary decision-making tool”. That is the organisation with the best possible access to the training distribution, reporting a true positive rate of roughly one in four.

The False Positive Problem and Who It Lands On

The TOEFL Study

GPT detectors are biased against non-native English writers, by Liang, Yuksekgonul, Mao, Wu and Zou, appeared in Patterns in 2023. The method was simple: run seven widely used detectors over 91 TOEFL essays written by non native English speakers and 88 essays written by US eighth graders.

The detectors handled the American schoolchildren accurately. On the TOEFL essays the average false positive rate was 61.3%. All seven unanimously flagged 19.8% of those human written essays as AI authored, and at least one detector flagged 97.8% of them.

Then the researchers did the thing that gives the game away. They had the same essays rewritten with more literary language, and the average false positive rate dropped from 61.3% to 11.6%.

Why the Error Lands Where It Does

That drop identifies exactly what is being measured. The tools detect limited lexical variety, which is a property of writing in a second language, and they report it as machine authorship. The 2023 benchmark found the same effect from another direction: human written text machine translated into English lost 20 percentage points of accuracy, so a researcher who drafts in their own language and translates is at raised risk of a false accusation.

What follows is not a close call. A tool with a 61.3% false positive rate on one identifiable group of writers must not be used to make a consequential decision about a person. The 2023 paper puts the practical version well: these tools return a percentage with no evidence attached, so a student accused on that basis alone “would have no possibility for a defence”.

Why Editing Defeats an AI Detector

The clearest experiment is Paraphrasing evades detectors of AI-generated text by Krishna, Song, Karpinska, Wieting and Iyyer. They built an 11 billion parameter paraphrase model called DIPPER and pushed generated text through it. Detection accuracy for DetectGPT fell from 70.3% to 4.6% at a fixed 1% false positive rate. Watermarking, GPTZero and OpenAI’s classifier were evaded as well.

Sit with the corollary, because it is the honest summary of the whole category. If unedited output is detected at 74% and edited output at 42%, then what the tools reliably identify is not machine authorship. It is carelessness. A page pasted straight out of a chat window gets flagged. The same page after a competent editor has cut a third of it, added two specifics only the author knows and rewritten the openings does not. The detector is a proxy for effort, and a poor one.

The Krishna paper’s proposed defence is worth noting because it is not detection at all. It is retrieval: search a database of 15 million previously generated sequences for a near match, which caught 80% to 97% of paraphrased generations while flagging 1% of human text. It works because it compares against a record of what was genuinely produced. It also requires the model provider to keep that record and run the check, which no public tool can do.

Watermarking, the Technically Serious Alternative

How It Differs

Watermarking intervenes at generation time rather than guessing afterwards. Google DeepMind’s SynthID for text sits in the sampling loop as a logits processor and, in Google’s own words, “augments the model’s logits using a pseudorandom g-function to encode watermarking information in a way that helps you determine if the text was generated by your model, without significantly affecting text quality”. The developer documentation describes a Bayesian detector returning one of three states: watermarked, not watermarked, or uncertain.

Google announced text watermarking for the Gemini app and web experience in May 2024. That is a real deployment, and it is a different kind of claim from a style guess, because the signal was deliberately inserted rather than inferred.

The Limits Google States Itself

The same documentation is candid about what breaks it. Watermarking “is less effective on factual responses, as there is less opportunity to augment generation without decreasing accuracy”, and detector “confidence scores can be greatly reduced when an AI-generated text is thoroughly rewritten, or translated to another language”. Google adds that SynthID “is not designed to directly stop motivated adversaries from causing harm”.

There is a parallel effort in provenance metadata rather than statistical signal. The C2PA Content Credentials specification, now at version 2.4, binds a cryptographically signed manifest to an asset recording its origin and the edits applied to it. It is explicitly opt in and global adoption is the stated goal rather than the current state.

Both approaches share one hard boundary. They tell you that a participating provider generated a particular string. They tell you nothing about output from a provider that does not participate, and they weaken under exactly the rewriting that a careful person performs anyway. Watermarking is a supply chain control for whoever runs the model. It is not a way to audit a stranger’s document.

Does Google Penalise AI Content?

Google’s position was published in February 2023 and has not moved. The Search Central post is called “Google Search’s guidance about AI-generated content”, the section heading is “Rewarding high-quality content, however it is produced”, and the load bearing sentence is this: “Our focus on the quality of content, rather than how content is produced, is a useful guide that has helped us deliver reliable, high quality results to users for years.”

The post also states that “not all use of automation, including AI generation, is spam”, noting that automation has long produced sports scores, weather forecasts and transcripts. What is a violation is using automation, AI included, with the primary purpose of manipulating ranking in search results.

What Actually Gets Penalised

The policy that bites is scaled content abuse. Google defines it as “when many pages are generated for the primary purpose of manipulating search rankings and not helping users”, aimed at “creating large amounts of unoriginal content that provides little to no value to users, no matter how it’s created”.

Read that final clause twice. Production method is explicitly irrelevant. Two hundred thin pages spun by hand from a town-name template violate the policy exactly as much as two hundred spun by a model. Google’s guidance on using generative AI content says the same thing from the other side: the technology is useful for research and structure, but using it “to generate many pages without adding value for users may violate Google’s spam policy on scaled content abuse”.

So there is no AI penalty to detect and no detector score that Google consults. The distinction is purpose and value. One page answering a question your customers genuinely ask, containing something only your business knows, is not abuse even if a model drafted the first version of it.

What to Measure Instead of a Score

Our own numbers make the point better than an argument does. Over a recent 28 day window this site drew 52,331 impressions from Google, with 531 queries sitting in the top ten positions. Those 531 queries produced 28,847 impressions and 161 clicks, a click through rate of 0.56% where positions four to ten normally earn between 2% and 8%. Four hundred and fifty two of them earned no clicks at all.

None of that is a detection problem. It is a differentiation problem, and no tool in this category measures it.

The metrics worth watching are click through rate against the position you hold, whether AI answers cite you or merely rank you, which we cover in why content ranks but never gets cited and in what Google AI Mode means for your traffic. Then the editorial decisions that follow: content pruning for pages that answer nothing specific, and content refresh for pages that once did. A technical SEO audit surfaces the mechanical half of the same picture, which is whether the pages you care about are even eligible to be seen.

What to Do If a Detector Flags Your Work

A Score Is Not Evidence

The number is a model’s opinion about style, produced with no access to any fact about how the text was made. That is not rhetoric, it is the design. Say so plainly, and cite the peer reviewed testing rather than arguing from your own credibility, because the research is on your side and your credibility is what is being questioned.

Provenance You Control

Version history is the strongest thing you have, and it only helps if it exists before you need it. Keep documents in a system that records revisions, keep your outline and notes, keep the sources you read and when you read them. A document that grew across forty saved revisions over nine days is a far better answer than any counter-score.

Ask for the same tool to be run on something you wrote years ago, before generative AI was available. That is precisely the control the 2026 study used. Ask whether a second tool agrees, because inconsistency between tools, and between repeated runs of the same tool, was documented in the 2023 benchmark. If English is not your first language, or you drafted in another language and translated, put the 61.3% and the 20 point translation penalty in front of whoever is making the decision.

Humanisers and the Economics of Beating a Broken Test

Paying a monthly subscription to defeat a test that is already wrong on 26% to 58% of the cases that matter is a poor use of money, and the 2023 benchmark recorded one tool in this space advising users to edit their text until there is less detectable AI content, which is a category arguing against its own premise.

There is a real cost beyond the fee. Humanising passes work by adding lexical variance, which in practice means thesaurus substitutions, hedges and inflated sentences. You end up optimising for a machine that does not influence your rankings, at the expense of the human who does decide whether to enquire. If you have budget for this, spend it on an editor.

Where an Agency Actually Adds Value

Not in dodging detectors. The work that changes outcomes is editorial process and provable expertise: a subject brief that identifies what your business knows that competitors do not, a named author with real credentials, sources cited and dated, and a review step where somebody who has done the job checks that the claims are true. That process happens to make the output undetectable, but that is a side effect rather than the objective.

Mecanik builds content programmes on that basis and audits existing ones through our SEO audit service, which starts from what your pages currently earn rather than from how they were written. If your pages rank and nothing happens, the fix is specificity and authority, and a technical SEO audit will tell you which of the two is actually blocking you. Detector scores will not appear in the report, because they measure nothing you can act on. If you want the wider commercial picture first, our guide to what an AI SEO agency does and what to pay covers where the money genuinely goes.

Frequently Asked Questions

Are AI detectors accurate? Not reliably. A 2023 study in the International Journal for Educational Integrity tested 14 tools over 756 individual tests and found every one scored below 80% accuracy, with only five above 70%. Accuracy was 74% on unmodified AI text, 42% once a human had edited it, and 26% after machine paraphrasing. A February 2026 study of Turnitin and Originality found macro-average accuracy of 0.61 and 0.69 respectively.

Does Google penalise AI-generated content? No. Google’s published guidance states that its focus is on the quality of content rather than how content is produced, and that not all use of automation is spam. What violates policy is scaled content abuse, defined as generating many pages primarily to manipulate rankings rather than help users, and Google states this applies no matter how the content is created.

Why do AI detectors flag non-native English writers? Because they measure lexical variety and predictability rather than authorship. A 2023 study in Patterns ran seven detectors over 91 TOEFL essays by non native speakers and recorded an average false positive rate of 61.3%, against near accurate results for US eighth-grade essays. Rewriting the same essays with more literary language cut the false positive rate to 11.6%.

Can an AI detector tell if text was edited by a human? Effectively no, and this is the largest gap. Accuracy on AI text with light human editing was 42% in the 2023 benchmark, and a 2026 study found near zero recall on hybrid documents built from roughly half human and half AI text. Research on paraphrasing dropped one detector from 70.3% to 4.6% at a fixed 1% false positive rate.

What should I do if I am wrongly accused of using AI? Treat the score as an opinion about style, not evidence, and say so citing the published research. Produce provenance you control: document version history, drafts, notes and sources with timestamps. Ask for the same tool to be run on work you wrote before generative AI existed, and ask whether a second tool agrees, since inconsistency between tools is documented.