Stop Wasting Money on AI Detectors: Why Statistical Guesswork Cannot Solve the AI Problem

This article was first written in 2024 on LinkedIn. This version has been expanded with additional research and updated examples.

A robotic hand reaching toward a glowing network of connected nodes — Detectors do not see authorship — they see statistical patterns, and patterns can mislead. Photo: Pexels

Artificial intelligence is now part of ordinary writing, research, coding, education, marketing, and office work. Students use it to brainstorm assignments. Researchers use it to organise literature and draft sections. Businesses use it to generate product descriptions, emails, reports, and customer support content. The concern about undisclosed AI use is legitimate.

The problem is the shortcut many institutions reach for: use AI to catch AI.

AI detectors are often marketed as if they can determine whether a document was written by a human or generated by a machine. They cannot do that with certainty. They produce statistical estimates based on patterns such as predictability, phrasing, sentence structure, and similarity to examples seen during training.

That difference matters. A plagiarism checker can point to a matching source. An AI detector usually cannot point to a verifiable act of authorship. It can only say that a piece of text resembles patterns it associates with AI-generated writing.

The evidence is not reassuring:

OpenAI retired its own AI classifier after noting its low accuracy.
Stanford researchers found that AI detectors were biased against non-native English writers, with many human-written essays incorrectly flagged as AI-generated.
Universities and educators increasingly warn against using detector scores as proof of misconduct.
Simple editing, paraphrasing, translation, grammar correction, or mixed human-AI workflows can change detector results.

So yes, students and professionals can misuse AI. But no, an AI detector should not become the final judge of honesty, authorship, or academic integrity.

Human review still matters. A teacher, editor, manager, or supervisor who knows the writer may notice a sudden shift in tone, vocabulary, reasoning, or structure. Even then, that is a clue, not proof. Writing style can change naturally as people read more, learn new material, use grammar tools, receive editing support, or adapt to a different audience.

The real question is not whether a detector can produce a score. The question is whether that score is reliable enough to justify consequences. In many real-world settings, the answer is no.

Examples of Obviously AI Content

AI content can be detected sometime for use of em-dashes, making every topic seem deep. However in most cases its very nuanced and mistakes by the writers are what get the caught as exemplified curated by Bilal Mushtaq on X, formerly known as Twitter.

When you review the papers, you will see that most have been flagged for leaving some standard boilerplate text and not for any other specific wording.

AI detectors pretend to detect a specific wording and hide behind the words "proprietary technology" without providing any detection methodology. Some word sequences may seem unnatural. Many trigger words have also been noted to trigger AI detectors despite such expressions being common in everyday language. But since language, in terms of vocabulary and style, varies from person to person, it is easy to attribute the oddities to other things rather than AI.

AI-generated and human-generated text are indistinguishable with asterisks. While you may be able to detect some AI content from a mile away, AI detectors have struggled, resulting in many false positives and negatives. That is because AI detectors try to guess how common human authors often write a sentence versus AI. Unfortunately AI-generators are not 100% correct in fact their accuracy is in the lower digits. As AI advances

Examples of False Positives and False Negatives

A classic example of AI detectors displaying their inconsistency is when they labelled the Bible as AI-generated. You can check the articles by Atheer Mahir below. He cites various studies on the topic, with each study further proving how unreliable AI detectors can be.

That distinction matters.

Many AI detector companies claim to identify AI-generated writing using proprietary technology. Yet few provide meaningful technical details about how their systems reach conclusions.

Most detectors analyze characteristics such as:

Sentence predictability
Writing consistency
Vocabulary patterns
Structural repetition
Statistical similarities to AI-generated text

They are not detecting authorship.

They are estimating probability.

This is fundamentally different from plagiarism detection.

A plagiarism checker can compare text against known sources and produce verifiable evidence. AI detectors cannot compare your document against every sentence ever generated by ChatGPT, Gemini, Claude, or any other AI model.

Instead, they attempt to answer a much harder question:

Does this text resemble content that AI systems often produce?

The answer is never certainty.

Only probability.

The False Positive Problem

AI detectors struggle with two types of errors:

False negatives, where AI-generated content is classified as human-written.
False positives, where human-written content is classified as AI-generated.

The second problem is significantly more dangerous.

A false negative may allow someone to bypass a policy.

A false positive can damage a student's academic record, cost a writer a client, or unfairly tarnish someone's reputation.

One of the most widely discussed examples involved AI detectors identifying passages from the Bible as AI-generated. Atheer Mahir documented several such cases while discussing the reliability of AI detection tools.

I conducted a similar experiment using ZeroGPT, one of the most popular AI detectors on the market.

I tested ZeroGPT, one of the most popular AI detectors in the market, with some of the pre-AI articles today, and the results were astounding. The first article I tested was a CNN article by Greg Krieg , and what do you know, his article was created 100% by AI. According to ZeroGPT, the article was 100% AI-generated. Given the year the article came out we all know that is impossible.

Yet the detector confidently produced a result that was demonstrably wrong.

ZeroGPT incorrectly identifying a 2016 CNN article as 100% AI-generated

Figure 1: A CNN article published years before ChatGPT existed was classified by ZeroGPT as 100% AI-generated. The result demonstrates how confidence scores can be mistaken for evidence.

The lesson is simple:

Confidence is not evidence.

An AI detector may produce a precise percentage, but precision should not be mistaken for proof.

Can Humans Detect AI-Generated Content?

A common misconception is that rejecting AI detectors means believing AI-generated content cannot be identified.

The reality is more nuanced.

Experienced educators, editors, researchers, and writers can sometimes identify signs that content may have been heavily assisted by AI. However, these signs are clues rather than proof.

According to the Wikipedia article on Artificial Intelligence Content Detection, common indicators include:

Repetitive sentence structures
Excessively uniform tone and style
Generic or superficial explanations
Hallucinated facts, citations, or references
Overly balanced arguments that avoid taking a position
Unusual confidence in incorrect information
Lack of personal experience or original insight

These characteristics are not unique to AI.

Human writers can produce them, and modern AI systems can often avoid them. Likewise, a skilled writer using AI responsibly may produce work that contains none of these indicators.

This is why experienced reviewers rarely rely on a single clue. Instead, they consider context, drafts, revision history, subject knowledge, and whether the author can explain the reasoning behind the work.

The challenge becomes even greater when content is partially AI-assisted rather than entirely AI-generated. A student may brainstorm with AI but write the final paper themselves. A journalist may use AI to summarize research but write the analysis independently. A researcher may use AI to organize literature while remaining responsible for the conclusions.

In such cases, the question is no longer whether AI was used.

The question becomes how it was used.

What AI Detector Companies Say

It may come as a surprise, but AI detector companies know that their technology is inaccurate.

Originality.ai founder Jon Gillham explicitly states:

"AI content detection is not perfect, and it does produce false positives."

Originality.ai further notes that results may be affected by:

Grammarly corrections
Academic formatting
Statistical data
Quotations
Public-domain content
Short documents

Consider what that means.

Many of these characteristics are indicators of quality writing.

Research papers should contain statistics. Journalistic work should contain quotations. Academic writing should contain citations.

If the same features that improve writing also increase the likelihood of false positives, then writers have little practical way of determining whether a detector's verdict can be trusted.

OpenAI's Failed Attempt at AI Detection

If anyone should be able to detect AI-generated content, it should be the company behind ChatGPT.

OpenAI launched an AI classifier in January 2023 and later retired it after acknowledging its limitations.

The company reported that the system correctly identified AI-generated content only 26% of the time while incorrectly flagging human-written content approximately 9% of the time.

Those numbers are hardly reassuring.

Today, OpenAI openly acknowledges that AI detection systems are unreliable, prone to errors, and easily influenced by minor edits.

OpenAI's current guidance recognizes several uncomfortable realities:

AI detectors are unreliable.
ChatGPT has no ability to identify whether content was generated by AI.
Detection systems are prone to error.
Small edits can significantly alter detector results.

Rather than relying on detector scores, OpenAI recommends reviewing drafts, revision histories, prompts, and evidence of the author's work process.

These are far more reasonable solutions.

The Academic World Is Moving On

When this article was originally written, many universities were experimenting with AI detection software in the hope that it would provide a reliable way to identify AI-generated work.

The evidence since then has pointed in the opposite direction.

Universities, educators, and academic institutions increasingly recognize that AI detectors produce false positives, struggle with reliability, and should not be used as the sole basis for disciplinary action.

In 2025, the University of Cape Town announced that it would discontinue the use of AI detectors because of concerns about reliability and the risk of falsely accusing students.

Similarly, admissions bodies and educational organizations have begun focusing on responsible AI use rather than attempting to ban AI outright. UCAS, the organization responsible for managing university admissions in the United Kingdom, now provides guidance on how students can use AI tools responsibly and ethically.

The conversation is slowly shifting from:

"How do we catch AI?"

to:

"How do we assess understanding in a world where AI exists?"

That is a much more productive discussion.

AI Is Becoming This Generation's Calculator

Critics of AI detectors are sometimes accused of defending cheating.

That misunderstands both the purpose of AI and the history of technological progress.

Throughout history, humanity has advanced by creating tools that extend our capabilities.

We invented writing so we would not have to memorize everything.

We invented books to preserve knowledge across generations.

We invented calculators so mathematicians, engineers, and scientists could focus on solving problems rather than spending hours performing repetitive arithmetic.

Few people today would argue that engineers should design bridges using only pencil-and-paper calculations. Few scientists would willingly abandon computers and return to hand-calculated statistical analysis.

Society accepted these tools because they increased productivity, reduced errors, and allowed human effort to be directed toward more meaningful work.

Artificial Intelligence represents the next step in that progression.

As human knowledge expands, the problems we attempt to solve become increasingly complex.

Modern medicine, engineering, climate science, economics, genetics, mathematics, and software development involve challenges that no individual can fully process without assistance.

AI is already helping researchers discover new materials, accelerate medical breakthroughs, analyze massive datasets, generate software, compose music, create art, and solve mathematical problems that would otherwise require significantly more time and resources.

The important question is not whether AI is being used. It already is.

The important question is whether the person using AI understands the work, can verify the results, and remains accountable for the final output.

A student using AI to brainstorm ideas is not fundamentally different from a student using a calculator to simplify arithmetic.

A researcher using AI to organize literature is not fundamentally different from a researcher using statistical software to analyze data.

The danger lies not in the tool itself but in using the tool without understanding the underlying concepts.

Attempting to ban AI entirely would be like banning calculators because some students once cheated on mathematics exams.

Such restrictions would not stop innovation.

They would merely slow it down.

The Future Is Verification, Not Detection

Even technology companies increasingly recognize the limitations of AI detection.

Rather than attempting to guess whether content was AI-generated, many organizations are exploring content provenance systems that record how digital content was created.

Industry initiatives such as the Content Authenticity Initiative (CAI) and the Coalition for Content Provenance and Authenticity (C2PA) aim to attach metadata and cryptographic signatures to images, videos, audio files, and documents.

This approach focuses on verification rather than speculation.

Instead of asking:

Does this look like AI?

The better question becomes:

Can the creator show how this content was produced?

No system will be perfect, but provenance offers a far more promising path than relying on statistical guesses about writing style.

A Tale of Distrust, Ruined Careers, and Much Worse

The road to hell is often paved with good intentions.

AI detectors were introduced to combat cheating, but their misuse has created a different set of problems.

Freelance writers have lost clients after their work was incorrectly flagged as AI-generated. Long-standing professional relationships have ended because a detector produced a suspicious score.

I experienced this firsthand when a client told me she had fired another writer after an AI detector flagged their work.

Ironically, the same client later demonstrated how easily detector scores could be manipulated by changing only a handful of words.

A document that scored 97% AI-generated suddenly became 100% human.

Nothing meaningful had changed.

Students have faced similar challenges.

Some have been required to defend themselves against algorithmic accusations despite having completed their work honestly. In several documented cases, students were effectively forced to prove their innocence because software suggested they may have used AI.

That is a dangerous precedent.

Academic integrity should be protected through evidence, not probabilities.

Conclusion

Artificial Intelligence is not going away.

The technology has already become part of research, software development, education, science, medicine, journalism, and creative work. Future generations will likely view AI tools the same way we view calculators, spreadsheets, and search engines today: essential tools for solving increasingly complex problems.

The challenge is not how to eliminate AI.

The challenge is how to use it responsibly.

AI detectors attempt to solve that challenge through statistical guesswork. Unfortunately, the evidence from researchers, AI companies, universities, students, writers, and even OpenAI itself suggests that these tools are far less reliable than their marketing implies.

A detector may provide a clue.

It does not provide proof.

As AI becomes more deeply integrated into society, trust will depend less on detection and more on transparency, accountability, and verification.

The future belongs not to those who avoid AI, but to those who learn how to use it responsibly while remaining accountable for the results.

Update (2026)

This article was originally published in 2024.

At the time, the evidence already suggested that AI detectors were unreliable and prone to false positives. Since then, several developments have reinforced that conclusion.

Most notably:

Universities such as the University of Cape Town have discontinued the use of AI detectors because of concerns about reliability and false accusations.
Educational organizations such as UCAS have shifted toward guidance on responsible AI use rather than attempting to prohibit AI outright.
AI has expanded far beyond text generation and is now routinely used in software development, scientific research, medicine, education, data analysis, and creative work.
Industry initiatives such as the Content Authenticity Initiative (CAI) and Coalition for Content Provenance and Authenticity (C2PA) have gained momentum as alternatives to statistical AI detection.

The central argument remains unchanged:

AI detectors provide probabilities, not proof.

Research and editorial assistance provided using OpenAI's ChatGPT.

References

Content Provenance

Content Authenticity Initiative
https://contentauthenticity.org/
Coalition for Content Provenance and Authenticity (C2PA)
https://c2pa.org/

Stop Wasting Money on AI Detectors: Why Statistical Guesswork Cannot Solve the AI Problem

Examples of Obviously AI Content

Examples of False Positives and False Negatives

The False Positive Problem

Can Humans Detect AI-Generated Content?

What AI Detector Companies Say

OpenAI's Failed Attempt at AI Detection

The Academic World Is Moving On

AI Is Becoming This Generation's Calculator

The Future Is Verification, Not Detection

A Tale of Distrust, Ruined Careers, and Much Worse

Conclusion

Update (2026)

References

Research and Background

Examples of AI Use in Academic Publishing

AI Detector Failures

AI Detector Company Statements

OpenAI and AI Detection

Higher Education and AI

Content Provenance

Related reading

Safe for Savers, Risky for Guarantors: The Changing Calculus of SACCO Membership in Kenya

Bengula Inc Company Profile: Data, Digital Visibility, and Practical Growth Advisory

Kenyan Treasury Bonds Demystified: The Anchor of Long-Term Wealth in Kenya