Plagiarism-sniffing Turnitin tries to find AI writing by students – with mixed grades
Claims 98% confidence, tests suggest a human touch will beat the teachers
Updated Turnitin, which touts itself as a maker of plagiarism-detecting software for academia, demoed on Tuesday what we're told is a tool capable of detecting AI writing.
Crucially, it boasted its machine learning software will flag up computer-generated cheating when it has at least "98 percent confidence" it is right. Be as that may, that's not the same as accuracy. A model can be supremely confident and still completely wrong most of the time.
Indeed, tests carried out by the Washington Post found the detection tool's performance lacking.
Turnitin was good at identifying AI-generated text when the whole input had been created by ChatGPT, but struggled with a mixture between human and machine-written sentences. It also incorrectly flagged parts of an essay written by a student as AI-generated.
Teachers are grappling with the trend of students turning to tools like ChatGPT to write essays and complete homework, and academic institutions are struggling to enforce policies regulating the use of AI-writing software.
Turnitin claims the features now integrated into its existing plagiarism-identifying products highlight AI-generated text when it has high confidence, though no word on actual accuracy. Given a claimed 2.1 million US educators use the code, it would be nice to see some of its working.
"Turnitin's technology has high accuracy and low false positive rates when detecting AI-generated text in student writing," Turnitin's chief product officer, Annie Chechitelli, declared in a statement. "To maintain a less than one percent false positive rate, we only flag something when we are 98 percent sure it is written by AI based on data that was collected and verified in our controlled lab environment."
- Universities offered software to sniff out ChatGPT-written essays
- Uni plagiarism site buckles under crush of last-minute essays
- Europol warns ChatGPT already helping folks commit crimes
It's not clear how Turnitin verified the accuracy of its software, nor how it performs predicting text generated by different language models.
The vendor has been criticized for pushing its tools onto customers without being transparent about how they work. Some top Russell Group universities in the UK – including Cambridge University, for example – have opted out of using Turnitin's AI detection tools, according to the Financial Times.
Predicting whether a passage of text was produced by AI is tricky. Even OpenAI, which built ChatGPT, warned that its own AI Text Classifier can only correctly identify text as likely to be written by AI 26 per cent of the time.
A group of computer scientists from the University of Maryland recently published a study that found it was easy to evade detection tools – even for text that had been watermarked.
Turnitin's AI-detection software is currently free to use.
"Educators told us that being able to accurately detect AI written text is their first priority right now. They need to be able to detect AI with very high certainty to assess the authenticity of a student's work and determine how to best engage with them," said CEO Chris Caren.
The Register has asked Turnitin for comment on validating the 98 percent figure. We'll let you know. ®
Updated to add on April 5
Turnitin got back to us to say, while ignoring most of our questions, that educators can't opt out of this new scanning system.
"No, there is no option to turn off the feature," a spokesperson told us, though added:
"Based on early conversations, we’ve made the exception to suppress AI detection for a select number of customers with unique needs or circumstances. Our goal is to be timely in enabling educators with tools that will inform them as well as guide us through this time of uncertainty, together. As we continue to have conversations with our customers, we will adjust our solutions to enable and reflect their evolving practices."
There were fears that Turnitin was running all submitted work through its AI detection model and just hiding the results from schools that opted out.
Editor's note: This article was updated after publication to add commentary on confidence versus accuracy.