Adam Day

There’s this phrase used by some technologists: ‘epistemic security’. Epistemic security has to do with things like the spreading of misinformation on social media. I.e. if we spend enough time on social media reading unreliable information, how does that influence us? That’s a matter of epistemic security. Epistemic security is interesting because if you start a conversation about epistemic security at a party, they don’t invite you back.

The Papermill Alarm

TL;DR: AI-generated-text detection ceased to be a good indicator of research fraud at the exact time that it became a good tool for generating fraudulent research. I remember finding ChatGPT usage in a research paper for the first time. It was April 2023, and a Twitter user pointed out that you could identify bots by their use of the phrase “as an AI language model…”. It’s a hallmark of ChatGPT usage.

Detecting genAI: beside the point?

TL;DR: The current version of the Papermill Alarm detects signals in 98.9% of the Hindawi retractions conducted over the last 12 months.** It’s never nice to see the harms caused by papermills, but it is good to see independent verification of the Papermill Alarm’s predictions.

Here’s a question: Why are there 2 multi-squillion-euro detectors at CERN’s Large Hadron Collider?

Predicting retractions with the Papermill Alarm

TL;DR:

The Papermill Alarm has grown to become a comprehensive mix of methods for detecting organised research fraud and simplifying a complex problem. But there’s something new. Something bigger and better from Clear Skies.

Bigger &amp; Better. Oversight from Clear Skies

This is

TL;DR: Join me at ConTech Live to hear about a recent project with Open Credo to see if we could detect unusual co-authorships in a dataset created by Anna Abalkina. Sign up here! Papermilling has a few definitions which you see here and there.

(The?) 3 kinds of papermills

TL;DR: There’s been a recent major upgrade to the private Papermill Alarm. It looks like we can predict the inclusion of journals on the Chinese Academy of Sciences (CAS) lists. You know how data scientists make predictions?

Cast a wide net

There’s a quote attributed to Ernest Rutherford: “That which is not measurable is not science. That which is not physics is stamp collecting”. I think his point was that a lot of scientific work is just

 documenting

things. In Rutherford’s day, there was a lot of exciting new

 creative

work happening in physics, so perhaps physics seemed special to him.

We need a chat about ChatGPT

Recently, I gave a presentation on the APIs for Papermill Detection offered by Clear Skies Ltd. I also touch on a newer service called the Clear Skies Standard Report. More on that in future posts… :)  Here’s the video:  https://medium.com/media/ee5a04aaed9fc53d2748e64516178ebe/href Would you like to know more?

Video: APIs for Papermill detection (ConTech API Series)

The first time I flew over the North Atlantic was quite an experience. Through the clouds, I could see some little white boats out sailing in the sea. It was puzzling: from 30,000 feet, those boats must have been huge for me to be able to see them at all.

How to find all the papermills (step 1)

This post is about The Papermill Alarm: an API for detecting potential papermill-products. There’s a field of study called ‘stylometry’ where we look at the statistical properties of someone’s writing and use that to model their ‘style’. People write in idiosyncratic ways.

The Papermill Alarm: Patterns in PubMed

The Papermill Alarm draws a line between papers which have the characteristics of papermill-products and those that don’t. Given data like that shown below, where red dots represent papermill-products and green dots represent normal papers, where should we draw that line?

Where to draw the line? Precision and recall

Last week, a paper I wrote on the subject of peer-review fraud was published in the journal

 Scientometrics

(free link here, preprint here)

 .

It was an interesting project to work on. I found a lot of examples where one referee would write a report during peer-review and then another referee would write an identical report in some other peer-review of some other paper.

Papermills in peer-review

Also… what is an API?

The Papermill Alarm API, is a service which you can send some article metadata to and which will return an alert telling you if the paper

 looks like

past papermill-products. Anyone can use it, but it definitely helps to have the support of an IT or data professional.

How to use the Papermill Alarm API

I once saw a brilliant presentation about how simple data analysis can detect credit card fraud**. The presentation showed a

 pattern

in how people use their credit cards. Given a large number of people who had been victims of credit card fraud, this pattern showed there was just 1 store in-particular where they had all used their cards. There was no

 observational evidence

of someone at that store stealing card details.

Patterns and Evidence (and how to find papermills before peer-review)

This is Ralph. How tall is Ralph? It seems simple, you could just hold a ruler up to the screen. But when you look at the ruler and use it to measure Ralph, are you actually measuring

 Ralph

, or are you measuring the ruler and using that as a

 proxy

? How accurate is your measurement? Are you including fur in the measurement? What if Ralph were to stand on his hind legs, like a mighty bear — how tall would he be then?

The definition of plagiarism

“Disruption” is what happens when a different technology develops to a point where it does what yours does

 well enough

to meet the needs of your customers. That means that, when customers switch, you can’t compete without changing your product completely. There’s a brilliant essay by Michael Nielsen from 2009 which predicted that disruption in academic publishing was imminent. It’s still worth a read, even now.

What luck

TL;DR: thousands of different authors use the same template emails to communicate with journals. Why?

So, we’ve established that papermills like to use templates. We see templates in referee reports and in the text of cookie-cutter research papers. There’s an important insight here:Templates are used in legit academic behaviour as well as in industrial research fraud.

Thank you for your warm work… earnestly

TL;DR: the rate of Papermill Alarm alerts is significantly higher in gold OA content than it is in subscription content. If you’re concerned about papermills, Clear Skies offers the Papermill Alarm: the world’s leading papermill detection tool. Get in touch for a demo. I’m a big believer in Open Science.

Papermills prefer Open Access

It’s 1 January 2021. Dozing in bed, floating in that warm &amp; fuzzy limbo between dreams and reality, and that’s where it hits him. That lightbulb moment. (At last!) He sits bolt-upright in bed, punches the air and shouts: “You know what, honey, I’m gonna start a papermill!”. She snorts awake; reluctantly conscious. Oh no. Not again.

 Not another hare-brained scheme…

“…and, by the end of next year, we’ll publish 20,000 fake papers!”.

Impossible to ignore

TL;DR: We can detect individuals with a high probability of being involved in milling papers. The question is: how should we respond? A few years ago, my bike was stolen. It was an organised job. The thieves arrived after dark, cut a hefty lock clean off, and my bike disappeared silently forever. I imagined it dismantled and sold off into some vast black market network. Poor bike. The police admitted that they weren’t going to do anything.

Keystone: detecting bad actors with network analysis

TL;DR: peer-review times appear to have been growing for a long time. The effects of COVID-lockdowns on peer-review are surprising.

A few months ago, I was invited to referee a research paper. So, I guess the editor thought that I was one of the 2 best people in the entire world to review this thing. That’s how it works, right? Flattered, I flagged the editor’s email to signal its importance and I got straight to work!

Is science getting slower?

Stories by Adam Day on Medium

<figure>
<img
src="https://cdn-images-1.medium.com/max/1024/1*ZCIW-42TiMHET6-AQOY-gA.png" />
<figcaption>Data supporting our work is gratefully received from a
number of sources including: Clear Skies’ publisher partners, the
Problematic Paper Screener, Retraction Watch, OpenAlex, ORCID, Crossref,
numerous sleuths including David Bimler, Guillaume Cabanac, Anna
Abalkina and others.</figcaption>
</figure>

**TL;DR:** The Papermill Alarm has grown to become a comprehensive mix
of methods for detecting organised research fraud and simplifying a
complex problem. But there's something new. Something bigger and better
from Clear Skies.

Do you know what an "SEP field" is? It's from Douglas Adams:

> An SEP is something we can't see, or don't see, or our brain doesn't
> let us see, because we think that it's somebody else's problem. That's
> what SEP means. Somebody Else's Problem. The brain just edits it out,
> it's like a blind spot.

I used to think the "SEP field" --- a fictional tool for making
something invisible --- was just a joke, but the more I think about it
the more I think it's a real thing.

I'd spent a chunk of COVID lockdown working out a scalable solution to
the papermill detection problem. Papermills were a small part of what I
wanted to do with Clear Skies, but it was already clear to me at that
time that this was going to be a big project. I made this solution into
an API (called "Papermill Alarm" available via RapidAPI) allowing
publishers to self-serve and screen their submissions for problematic
content. The Papermill Alarm was the first commercial service
*dedicated* to papermill detection (but it's probably worth mentioning
that it wasn't the first papermill detector that we released --- [that
was the duplicate submission check in the Clear Skies Article
Tracker](https://medium.com/@clearskiesadam/how-to-find-all-the-papermills-step-1-f00c624facb1)).

Some publishers were quick to hop onto the API and start using it. By
and large, it was used responsibly and it felt great to know that the
service was helping people. Sadly, there was a small amount of abuse of
that service by papermills (and even by one publisher), so the free
version was short-lived, but I think it showed that papermill detection
was a tractable problem.

The whole time, there was something in the back of my mind.

It must have been at least a decade ago. I was editing a journal and I
saw a referee make the best catch I've ever seen. A manuscript was sent
to him for review and he responded with a copy of a published paper
written in his native language. He pointed out that

- the published paper was identical to the manuscript I'd sent him
 except the one I had sent him had been translated to English and
- the original, which had been published some time previously, had
 different author names on it.
- So it looked like someone had simply translated a paper, put their own
 name on it and sent it off for review hoping to get a paper published
 without actually doing any research.

It was really odd, but the team handled it like your typical plagiarism
case. The matter was investigated & authors asked to explain. When they
couldn't, the manuscript was rejected, and the authors were warned off
repeat behaviour.

Even after it was all finished, something was bothering me, but it went
to the back of my mind.

Hop forward to late-2022 and it was clear that there was demand for the
Papermill Alarm, but that early version needed to develop.

[I've always viewed misconduct detection as a network analysis
problem](https://medium.com/@clearskiesadam/papermills-in-peer-review-c507b12aa4ac).
Indeed, it turns out that network analysis is the primary analytical
method employed for fraud detection in other industries, too.

One method that was in constant development from shortly after the
Papermill Alarm's initial release was 'Keystone'. [Keystone is a network
analysis that finds the individuals most-connected to research
fraud](https://medium.com/@clearskiesadam/keystone-papermill-detection-with-network-analysis-3a48de927406).
The choice of name isn't an accident. The keystone in a building is
usually the one that holds it all together --- if you remove a brick
from a structure, it will probably be ok, but remove the keystone and it
all falls down. I think that these individuals are structurally
important to the networks of bad actors that perpetuate research fraud.
If we're going to point limited resources at investigations we should
target the dealers, not the users. So this shows where we might focus
our efforts.

By early 2023, misconduct detection was clearly in-vogue. So we expanded
the tool to cover all areas of science as well as adding new detection
pipelines and an introductory version (called "Papermill Alarm: Public")
for cases where data was limited.\*\*\*

Then I heard a story.

An author had submitted a paper to a journal, only to be accused of
plagiarism. The journal had already received an identical paper with
different authorship. Classic plagiarism case, right?

But the author protested and insisted that the paper was their own
original work. They hadn't shared it, *but they had written the work in
their native language, and then sent it to a translation company to be
translated to English.* So it seemed that the paper had been translated
by the translation company and that the translation company had then
sold the manuscript to different 'authors' before the real authors had
the chance to publish it.

Isn't that awful?

And that was what had been gnawing at the back of my mind for all those
years. *It's too much work, isn't it*? If someone couldn't be bothered
to write their own paper, why take the time to translate a paper, why
not just make something up or copy/paste something already in English
(this was before the days of plagiarism detection)? There had to be an
intermediary. Someone who was already doing the translation work anyway
who could gain further by selling the manuscript.

But the other thing was realising that, despite all that time spent
studying papermills, I had encountered one and not even realised it.\*\*

I learned some valuable lessons there.

- First: in a plagiarism case, it is absolutely possible for the fake to
 be published before the original.
- Second: it was very easy for me to focus on my own remit --- my job
 was to manage the journal and deal with occasional research fraud
 cases. It wasn't my problem to understand the mechanisms by which that
 fraud was happening industrially, that's an SEP.

I think that that's one reason why the scale of the papermill problem
was never really understood until recently --- we only ever saw one
tentacle at a time and it was no one's job to tug on them. Papermills
are great at hiding in plain sight.

[By the end of 2023, the scope of the Papermill Alarm was comprehensive.
We had unique Oversight of the whole problem. Its history, its methods,
and scale.](https://www.nature.com/articles/d41586-023-03464-x)

<figure>
<img
src="https://cdn-images-1.medium.com/max/549/1*S6yP5iDlszTWs9OI5486-g.png" />
<figcaption>By February 2024 we had incorporated a number of new checks
into the Papermill Alarm API.</figcaption>
</figure>

Interestingly, we had also already built a method which could --- quite
by accident --- detect the behaviour described above. Again, the
duplicate check in the Article Tracker would potentially catch a case
like this at the moment of submission (but it would still be up to the
investigator to work out which paper was the original).

One key thing here is that, as we see patterns emerging that 'plain
sight' part becomes acutely obvious.

<figure>
<img
src="https://cdn-images-1.medium.com/max/1024/1*S437zT85OUIDwLpQi-qYpg.png" />
<figcaption>Version 1 of the Papermill Alarm. This image was made around
the start of 2023 (we get better accuracy now). I haven’t shared it
publicly before because I was concerned it could be harmful. At this
point, though, I think the issues at Hindawi are widely known. That
said, if I ever needed to convince anyone that the Papermill Alarm
worked, this image was enough. It’s clear that the growth coincided with
the alerts and that the stream of retractions confirmed our
predictions.</figcaption>
</figure>

We want to give you Oversight of these patterns and that's why we're
delighted to launch *Oversight,* Clear Skies' primary data analytics
service.

Oversight has been in development for some time. It was first made
available in mid-2023 for a few bespoke use-cases. It was then upgraded
to full industry-level coverage. I'd like to extend my thanks to
everyone who gave feedback on the service. We're very grateful for
your support.

There's a lot coming here. But let's come back to it in another post.
For more information, [contact us](https://clear-skies.co.uk/contact/).

#### Addendum

\*\*My definition of 'papermill' is 'organised research fraud' or, more
nerdily, 'any repeatable pattern of fraudulent behaviour'. So, commit
fraud once and that's bad, but do it twice and you're on the radar. So a
translation service that sells manuscripts constitutes a papermill.

\*\*\*I think that Clear Skies can claim the first 'detection' of a
ChatGPT-generated paper ([I'm not sure if 'detection' is the right
word --- it was just a Google
search](https://x.com/ClearSkiesAdam/status/1647861389573103617)!). But
that was one route we didn't go down. [Technical solutions for
genAI-text detection are dubious at best and, even at that time, it was
clear that the legitimate use-cases for genAI were going to eclipse the
fraudulent ones](https://medium.com/p/01f6c3d8e05c). I think there are
useful applications for genAI-related fraud detection, but it's nuanced.

![](https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=85e51803cbfa){width="1"
height="1"}