The first rule of research integrity is always cite your sources. In fact, if there was only 1 rule, that would be it. Almost all research misconduct boils down to someone trying to take credit for work, discoveries, achievements, or ideas that aren’t their own.
Clear Skies’ data analysis shows that peer-review rejects papers where we flag concerns. When I buy something with a credit card, I tap my card on a card reader to make a payment. It’s very convenient. Every now and then, I might go into a shop where I’ve never been before. Then, when I tap my card, I am asked to enter my Personal Identification Number (PIN) to verify that I am me. Why does this happen?
Clear Skies is making genAI detection with Pangram available to subscribers in Oversight. We’re excited to see how this helps users to investigate research papers.
Clear Skies created the first index of research integrity: Oversight. We offer metrics describing research standards covering the entire research ecosystem. If you want to know more about metrics for your portfolio, get in touch. I sometimes get asked why I picked the name ‘Clear Skies’. I used to fly gliders. Gliders don’t have engines, so technically, they don’t actually “fly”, they just kind of fall out of the sky.
This post provides an update on some recent new features in Oversight and walks through an investigation. Given the pace of development of Oversight, I might update it in future as things change. The problem is time. It’s time spent on misconduct investigations. Such investigations are often complex and sometimes inconclusive — fraud is intentionally hidden after all.
Oversight, by Clear Skies, is the first complete index of journal integrity. Oversight takes complex data on research integrity and enables a straightforward and intuitive understanding of every journal in the world. We rank journals based on analysis of data from our extensively field-tested integrity tools.
TL;DR: The Papermill Alarm has grown to become a comprehensive mix of methods for detecting organised research fraud and simplifying a complex problem. But there’s something new. Something bigger and better from Clear Skies.
“Disruption” is what happens when a different technology develops to a point where it does what yours does well enough to meet the needs of your customers. That means that, when customers switch, you can’t compete without changing your product completely. There’s a brilliant essay by Michael Nielsen from 2009 which predicted that disruption in academic publishing was imminent. It’s still worth a read, even now.
It’s 1 January 2021. Dozing in bed, floating in that warm & fuzzy limbo between dreams and reality, and that’s where it hits him. That lightbulb moment. (At last!) He sits bolt-upright in bed, punches the air and shouts: “You know what, honey, I’m gonna start a papermill!”. She snorts awake; reluctantly conscious. Oh no. Not again. Not another hare-brained scheme… “…and, by the end of next year, we’ll publish 20,000 fake papers!”.
TL;DR: AI-generated-text detection ceased to be a good indicator of research fraud at the exact time that it became a good tool for generating fraudulent research. I remember finding ChatGPT usage in a research paper for the first time. It was April 2023, and a Twitter user pointed out that you could identify bots by their use of the phrase “as an AI language model…”. It’s a hallmark of ChatGPT usage.
TL;DR: The current version of the Papermill Alarm detects signals in 98.9% of the Hindawi retractions conducted over the last 12 months.** It’s never nice to see the harms caused by papermills, but it is good to see independent verification of the Papermill Alarm’s predictions. Here’s a question: Why are there 2 multi-squillion-euro detectors at CERN’s Large Hadron Collider?