While teachers struggle with AI integration in classrooms, and the debate over whether AI can make people’s cognitive function decline continues, AI is generating a specific problem in the world of scientific research.
Olaf Zawacki-Richter studies educational technology. Last week, Zawacki-Richter, who describes himself as a “simple professor,” discovered a weird link on his Google Scholar profile. It attached his byline to an article that he had never worked on — which was then referenced in 72 other articles.
Download the Straight Arrow app today to get the stories that matter free from manipulation, bias or agenda.™
Point phone camera here
It might sound like a simple mistake, but it highlights a serious problem: People using generative AI to churn out research papers can make up fake citations that sound real, contaminating researchers’ online profiles.
Zawacki-Richter posted his discovery on LinkedIn, and was shocked when his post went viral, receiving 200,000 views well beyond his own network.
“It’s really a burning issue at the moment, every, everywhere,” Zawacki-Richter told Straight Arrow. Nature reported in April that there could be tens of thousands of hallucinated citations online, “polluting the scientific literature.”
Why does this matter?
“The first appearance of this fake citation was in an Indonesian journal in 2024,” Zawacki-Richter said. But it has since been replicated and repeated in other articles, demonstrating how a small piece of misinformation can spread easily online.
The paper would not catch the attention of a reviewer. It is called “Digital transformation in higher education: A systematic review.” This is “a topic I’m usually writing about. It has co-authors that I’m publishing with,” he said. Someone reading the paper on a list could easily miss that it was a pure invention.
“It really acquired a scholarly life on its own, because it was not just published, but it also got distributed in other papers,” Zawacki-Richter told Straight Arrow.
“That is really scaring me that this can happen, that it’s not going to be published, but it gets cited so often.”
The fact that it is extremely easy to generate fake citations and have them appear in Google Scholar is a problem for archiving and understanding research impact, Zawacki-Richter said. “The strongest currency is citations, impact measured by citations,” he said. “And this system is inflated at the moment.”
What is Google Scholar?
Google Scholar is basically a search engine for academic research — kind of like a social media platform, but with links to papers instead of cat photos. It shows how authors rank based on how many other papers have cited them. This allows academics and regular people alike to see how different topic areas are linked, and understand whether to trust each other’s research.
The new paper and its subsequent citations have artificially inflated Zawacki-Richter’s online ranking, he said. And that makes it harder to trust the online rankings of other academics.
Having a fake citation also means that an entire research paper is called into doubt.
“It was cited in a context of the text that does not make sense,” Zawacki-Richter said. “It can mean that the whole text of these articles is fabricated, not only the reference.”
Generative AI in academia
The fake paper being attached to a real profile highlights a problem with how the Google Scholar platform works; it scrapes journals for references, and includes them — even if the included URL is a fake address. The introduction of generative AI and the ease with which fake papers are being created means that the platform needs to update its policies, according to Zawacki-Richter.
The platform records articles from all scientific journals — including ones that are “pay-to-play.” Zawacki-Richter is concerned that some of these fake citations are being propagated by “predatory” journals that accept money from researchers who desperately want to be published — anywhere — and do not do sufficient fact-checking or review.
We need “a new quality assurance mechanism there that validates references,” he said. “There has always been plagiarism and bad academic practice. But now it can be kind of industrialized, scaled up.”
Round out your reading
- All your questions about napping, answered.
- Trump’s $5,000 promise echoes past payouts that never materialized.
- Inside the effort to make data centers pay their share of electricity costs.
- Why did the Feds seize the ’largest Martian meteorite on Earth’?
- Photos and video show exactly where and when Trump visited Ground Zero after 9/11.
