OpenAI has developed a text watermarking tool capable of detecting essays authored by ChatGPT with significant accuracy. This development follows a report from The Wall Street Journal suggesting the tool is ready for release but has been withheld due to ongoing debates within the company.
The detection method, known as text watermarking, involves subtly altering the way ChatGPT selects words, embedding a kind of invisible watermark that can be identified later. This method aims to trace the origins of the text back to ChatGPT specifically, distinguishing it from content generated by humans or other AI models.
Challenges with the Watermarking Tool
Despite the promising nature of this technology, OpenAI has expressed concerns over its potential limitations and risks.
The tool has proven to be highly accurate under specific conditions, as noted in an update to a May blog post on OpenAI’s website, which TechCrunch reported on. However, the tool struggles with texts that have been heavily modified — for instance, through translation software, rephrasing with another AI tool, or by inserting and then removing special characters within the text.
OpenAI’s research suggests that while the tool is effective against minor edits or localized tampering, such as paraphrasing, it is less reliable against more comprehensive changes, referred to as “globalized tampering.” These vulnerabilities could allow the detection method to be easily circumvented by those intent on cheating.
Furthermore, there are ethical considerations related to the deployment of this technology. OpenAI is particularly concerned about the potential for the tool to disproportionately impact non-English speakers, fearing it could stigmatize their use of AI for writing assistance.
In light of these challenges, OpenAI has not limited its research to watermarking alone but has also explored other methods like classifiers and metadata analysis as part of their extensive research into text provenance.
Featured Image courtesy of MirageC/Getty Images
Follow us for more updates on ChatGPT.