A research team from the University of Tübingen has developed a new method for identifying AI-generated text in scientific abstracts, finding that at least 13.5 percent of biomedical publications in 2024 may contain AI-written sections. Dmitry Kobak and colleagues analysed word usage in more than 15 million biomedical abstracts published between 2010 and 2024, demonstrating that certain words began appearing with unusual frequency following ChatGPT's introduction in November 2022.
AI tools tend to use certain words — like "delves," "crucial," "potential," "significant" and "important" — far more often than human authors do. In 2024, there were a total of 454 words used excessively by chatbots, and based on the frequency of AI-favoured words, researchers calculate that at least 13.5 percent of all biomedical abstracts appeared to have been written with chatbot assistance. Among authors from certain countries writing in less selective journals, as many as 40 percent of abstracts were AI-generated. Previous MIT Technology Review research indicates that AI-text detection tools are easily fooled, identifying only 42 percent of slightly modified ChatGPT-generated texts accurately.
The study's findings highlight that despite current limitations in AI-text detection technologies, analysing word usage patterns can effectively monitor scientific integrity. The increased use of the 454 identified words clearly correlates with ChatGPT's November 2022 release, suggesting widespread adoption of AI-assisted writing within the scientific community. Researchers found that detection tools identify original ChatGPT texts with 74 percent accuracy, dropping to 42 percent when text slightly modified.
Sources:
1.
2.
Rhiannon Williams
