AI-generated text is quickly becoming a problem in a variety of venues, most notably those involving education. While traditional plagiarism detectors still aren’t capable of differentiating between human writing and writing from AI programs, the creators of ChatGPT and its ensuing scourge upon the land are working on something that can tell the difference.
The operating word there is “working”, because the product–OpenAI’s AI Text Classifier–isn’t quite able to detect AI-generated content at a high enough rate to be put into practice; however, as with all AI products, one can assume that its existence implies that it will improve along the way.
Search Engine Journal reports that AI Text Classifier was built to “limit the ability to run automated misinformation campaigns, use AI tools for academic fraud, and impersonate humans with chatbots” according to OpenAI’s pertinent announcement.
Unfortunately, AI Text Classifier has a fairly low rate of success at this time, correctly diagnosing text written by AI only 26% of the time–and incorrectly attributing actual human writing to AI a whopping nine percent of the time.
Additionally, Search Engine Journal’s Matt Southern observes that the tool can be fooled with minor fixes to entirely AI-generated content, many of which can be implemented automatically through spell-checkers like Grammarly.
“Ultimately, the AI Text Classifier can be a valuable resource for flagging potentially AI-generated text, but it shouldn’t be used as a definitive measure for making a verdict,” concludes Southern.
For their part, OpenAI is forthcoming about the tool’s current weaknesses. “Our classifier is not fully reliable…It should not be used as a primary decision-making tool, but instead as a complement to other methods of determining the source of a piece of text,” reads their website.
OpenAI similarly lists concerns about AI Text Classifier’s success with shorter pieces, clarifying that it needs at least 1000 words before it can work accurately; it also cannot handle things like languages other than English, and its assessment of code-writing is profoundly unreliable.
Like ChatGPT, AI Text Classifier is free to use with an account. As something that will ostensibly get better with increased use, it would be to the benefit of humanity at large to consider this tool a necessary addition to any plagiarism prevention arsenal–albeit with the understanding that, at this time, it cannot stand on its own.
