As technology firms move fast and break things in the race to build artificial intelligence tools, they’re buying rare books in bulk, chopping off their bindings and feeding their pages through industrial scanners.
Human-created literature dating back at least a century is then used by the tech firms to train AI models and, ultimately, dictate chatbot responses.
Now, a technology company that helps crack down on students who use AI to cheat on homework assignments says it plans to use those same essays and worksheets to train AI.
Download the Straight Arrow app today to get the stories that matter free from manipulation, bias or agenda.™
Point phone camera here
The announcement underscores how technology companies are gobbling up massive amounts of human-generated writing — from newspaper articles to Reddit posts — to build AI information pipelines.
Education technology company Turnitin has announced updates to its end-user license agreement that will require students and educators to consent to having their data “used to train AI models,” company records show. Its precise plans for AI development are unclear. An FAQ on the company’s website notes the data will be used for “improving or developing AI tools and other products, features or services.”
The company has provided anti-plagiarism software to schools since the 1990s and rolled out its AI-detection tool in 2023.
Turnitin notes the changes come at a time when “the academic landscape is evolving with the use of AI” and “reflect the expansion of Turnitin’s AI tools.”
But the move is stirring controversy in academic circles. Some educators already had expressed concerns that tech firms are increasingly scraping research papers and other educational materials to train AI — often without consent from the original authors.
University librarian Ian Linkletter, an outspoken critic of AI-led efforts to thwart cheating, said Turnitin’s use of student and educator inputs could include “historical data.”
“My biggest concern about this is that I don’t believe students consented to this all the way back to the ’90s when they first started collecting student data,” Linkletter, a librarian at the British Columbia Institute of Technology in Canada, told Straight Arrow. “And I don’t think they’re consenting to this now, which is probably why they went to roll out this new user agreement.”
The updated user agreement was proposed to “provide a clearer agreement on AI tool use” to students and educators as it expands “in-product AI use,” a Turnitin spokesperson told Straight Arrow in a statement. Following pushback from the education community, the spokesperson said, the company “paused implementing” the user agreement until 2027 “to give our customers additional time to review.”
‘Students retain ownership’
Students routinely turn to chatbots to help with schoolwork despite widespread concerns that it is harming their cognitive abilities, recent research indicates.
Nearly two-thirds of students 12 to 29 years old report using AI to help complete homework assignments, according to a recent Rand Corporation survey. Yet a similar share says that the more students rely on technology for their schoolwork, “the more it will harm their critical thinking skills.” By and large, the students didn’t believe their AI-use constituted cheating.
As colleges and universities turn to tools like Turnitin to crack down on students’ AI use, it has pitted young people against the institutions they pay to attend. In at least one instance, a lawsuit successfully challenged a university’s allegation of cheating.
In February, a history student at Adelphi University in New York successfully sued the institution after it accused him of using AI to write a paper during his first semester of college. The Turnitin AI detector flagged the essay as being fully AI-generated, but the student ran the copy through two other AI-detectors which found the assignment was written by a human, according to court records.
A judge ruled the plagiarism allegation was without merit and ordered the university to remove the plagiarism charge from the student’s academic record.
Turnitin’s updated user agreement places the responsibility of thwarting AI cheating on the educators themselves. While its AI writing detection reports “are decision support tools,” the FAQ notes, they do not provide “conclusive findings” and teachers’ own “professional judgment remains the final authority” in judging cheating allegations.
Turnitin documentation notes that “students retain ownership of their submissions,” that the company does not “use the concepts or ideas” included in submissions and that it “does not sell or rent personal information.” Data owned by the company, it notes, is “aggregated, anonymized and/or deidentified.”
Linkletter, the university librarian from Canada, notes the company FAQ “makes it sound like they’re already doing AI training, which was very alarming,” in part because the language around how the company uses the information is “pretty open-ended” and encompasses “new products generally.” Linkletter’s institution has already opted out of using Turnitin’s AI detection software, he said.
“New products could mean anything at all, and it could also mean licensing the data to a third party,” Linkletter said. Although the documentation says Turnitin doesn’t rent or sell personally identifiable data, Linkletter said that could change and that chatbot developers like OpenAI and Anthropic may have an interest in the information “if they felt it could give them access to student data.”
“Turnitin is sitting on a gold mine,” Linkletter said, as AI companies place a premium on training materials written prior to commercial availability of chatbots. “They’re sitting on a huge database of pre-ChatGPT works and they seem to have taken the legal point of view that they’re entitled to use these works essentially for any purpose they want.”
Pushback against Turnitin is growing
Linkletter isn’t the only academic raising an alarm about the changes. The University of California, Berkeley, noted in a post on its website that instructors and students would be required to accept the Turnitin user terms despite an agreement between the institution and the company prohibiting the firm from using university information in AI systems.
A historian at the University of Southampton in England wrote on Bluesky that the institution was “ending its contract with Turnitin” over the company’s decision to “use student submissions to train AI models.”
“Institutions can and should resist this stuff,” Charlotte Lydia Riley, an associate professor, posted on the social media platform. She noted the anti-cheating software routinely flagged her students’ authentic work as plagiarism.
Another British institution, the University of Sussex, noted on its website that the change has raised “significant questions across the UK higher education sector regarding data privacy and institutional contracts.”
While the new Turnitin user agreement was supposed to go live on July 14, according to the institution, the company paused its rollout temporarily and will “work in collaboration with customer institutions to thoroughly address outstanding questions and provide greater confidence in any future changes.”
Linkletter said he hopes awareness about the issue will prompt the company to think twice about using the data to train AI tools. He told Straight Arrow he has a “sense of a public duty to warn people about unethical companies, especially when students are in danger.” In this case, students routinely share “very personal things” in classroom assignments and, although the data are anonymized, they could ultimately be reconstructed to identify individuals.
“I think it’s a huge privacy violation, potentially, for students to have those stories and experiences turned into an AI product,” he said. “Knowing what I do about AI and its propensity to leak parts of its training data if you prompt it correctly, I think there could potentially be a way that people could extract those stories.”
Round out your reading
- Former Fetterman staffers create social media account to expose former boss.
- Why Gen Z finally wants Boomer hand-me-downs.
- Want a copy of Pete Hegseth’s driver’s license? It’s for sale on the dark web.
- Smart glasses helped a man see the world. The privacy panic took it away.
- Americans are keeping cars longer, so the auto industry is making it easier to pay for repairs.
