A Strange Phrase Keeps Turning Up in Scientific Papers, But Why? : ScienceAlert

In recent months, a strange term known as “vegetative electron microscopy” has been making appearances in scientific papers, sparking curiosity and confusion among researchers. This term, although technically nonsensical, has managed to embed itself in our digital ecosystem like a fossil, thanks to the workings of artificial intelligence (AI) systems.

The origins of “vegetative electron microscopy” can be traced back to an unusual convergence of errors. It all began when two papers from the 1950s were scanned and digitized, inadvertently combining the words “vegetative” and “electron” from different sections of text. This mishap resulted in the creation of the phantom term, which has since found its way into Iranian scientific papers due to a translation error between Farsi words for “vegetative” and “scanning.”

As a result, “vegetative electron microscopy” has appeared in a growing number of publications, with 22 papers currently featuring the term according to Google Scholar. This has led to retractions and corrections from prominent publishers like Springer Nature and Elsevier, highlighting the challenges posed by errors perpetuated through AI systems.

Uncovering AI Contamination

To understand how “vegetative electron microscopy” has proliferated, researchers delved into the inner workings of modern AI models. By testing various language models, such as OpenAI’s GPT-3, it was revealed that the term consistently appeared in generated text. This contamination was found to persist in newer models like GPT-4o and Anthropic’s Claude 3.5, indicating a deep-rooted presence in AI knowledge bases.

The primary source of this error was identified as the CommonCrawl dataset, a massive repository of internet pages used to train AI models. The sheer scale of these datasets, combined with the lack of transparency in commercial AI models, poses significant challenges in identifying and rectifying such errors.

Implications for Knowledge Integrity

The case of “vegetative electron microscopy” underscores the broader implications for knowledge integrity in an era dominated by AI-driven research and writing. Publishers have grappled with how to address such errors, with responses varying from retractions to justifications.

Furthermore, the rise of AI-generated content has introduced new complexities in the peer-review process, with automated tools struggling to differentiate between legitimate research and convincing nonsense. Efforts to flag such content, like the Problematic Paper Screener, are limited in addressing unknown errors lurking within AI systems.

Ultimately, the prevalence of “digital fossils” like “vegetative electron microscopy” highlights the challenges of maintaining reliable knowledge in an increasingly AI-driven world. Transparency from tech companies, innovative research methodologies, and improved peer review processes are crucial in mitigating the impact of such errors on scientific discourse.

Aaron J. Snoswell, Research Fellow in AI Accountability, Queensland University of Technology; Kevin Witzenberger, Research Fellow, GenAI Lab, Queensland University of Technology; and Rayane El Masri, PhD Candidate, GenAI Lab, Queensland University of Technology

This article is republished from The Conversation under a Creative Commons license. Read the original article.