US Declaration of Independence Flagged as AI-Generated by Detectors

The integration of artificial intelligence (AI) into our daily lives has certainly brought with it many benefits. As helpful as this technology has been, however, it has brought its fair share of problems, as well. From political “deepfakes” to students automating their homework, AI detectors – themselves artificial intelligence programs – have become a crucial tool. But how accurate are they, really? The Declaration of Independence, written more than two centuries before such technology existed, was recently flagged by one such program as 98% AI content.

AI Is Not Infallible

Today, many instructors rely on AI detectors to see if students have used tools to write their papers. But, according to Christopher Penn, chief data scientist at Boston-based marketing analytics firm Trust Insights, “AI detectors are a joke.”

“What led me to the testing of AI detectors was seeing colleagues battling back and forth, arguing about whether a piece of content was AI-generated,” he told Decrypt. “I saw this on LinkedIn; some people were lobbing accusations against each other that so-and-so was being a lazy marketer, taking the easy way out, and just using AI.”

To prove his point, Penn decided to test the US Declaration of Independence and was appalled to discover that one detector claimed that 97.75% of the preamble was AI-generated.

So I decided to conduct my own experiment.

Using Federalist Papers No. 1, I plugged it into Grammarly’s plagiarism and AI text check. The results came back saying 100% of the document matched external sources and no AI-generated text was detected. That was encouraging. Next, I tested it in ZeroGTP’s detector and was told the text was AI/GPT generated: 93.74%.  Considering Alexander Hamilton published this first of 85 essays on October 27, 1787 – 237 years ago and long before AI existed – this was very discouraging.

AI programs can be used to rewrite content, so I decided to continue my experiment to see if rewriting No. 1 would be flagged in the detectors. Using ChatGPT, I copied and pasted the document and told the tool to rewrite it for me. Using the AI-generated rewrite, I plugged it into Grammarly’s detector. Keep in mind, this is 100% AI-generated, but Grammarly found “no plagiarism detected” and “28% of your document appears to be AI-generated.” Even worse, ZeroGPT told me: “Your Text is Human written” and that only 13.61% was AI/GPT generated.

Will Artificial Intelligence Replace Human Writing?

One of the concerns about AI is that it will replace human writing, but is this a valid fear? UC Santa Barbara English professor Sown Park explained that the two have different motives and are not in competition with each other. “People write to make sense of things,” she said. “The need to create and make meaning through oneself is constant. AI writing can only be [considered] better if we invalidate the joy people get from writing.”

Writing connects us to ourselves, the world, and each other, she explained. “It is only by putting it into words that I make it whole,” Park said, quoting Virginia Woolf’s autobiographical work, Moments of Being. Furthermore, literature does not progress over time and it is not a competition. “One cannot compare Woolf to Hemingway or Austen to Shakespeare – every writer’s job is to write what is true for them.”

To test this theory, I instructed ChatGPT to: “Write a short love poem.”  This is what it came up with:

In your eyes, the stars align,
A universe where hearts entwine.
Your touch, a breeze, soft and sweet,
The place where love and forever meet.

In every word, in every sigh,
You are my earth, my sea, my sky.
A love so pure, a light so true,
My world begins and ends with you.

Is it romantic and unique? I’ll let you be the judge.

Programs such as ChatGPT can be useful in getting ideas and suggestions for writing topics and points of interest, but they should not be used to create full documents or job resumes. It’s not just the moral aspect of cheating; these tools are not infallible. In fact, ChatGPT has a disclaimer warning users to always check important facts. Still, more and more people are relying on AI to take some of the load off their busy schedules.

And just for giggles, I tested this article, which was 100% written by a human or otherwise cited. Lo and behold, both Grammarly and ZeroGPT agreed with me; no AI-generated text was used. At least they finally got something right.

Takeaways:

1. More people are relying on AI to do their work for them.
2. Some people use AI detectors to catch people using AI instead of writing for themselves.
3. Detectors aren’t very reliable – they’ve been known to flag centuries old documents as AI-generated.