Leiden Madtrics

Over the past two years, the Information &amp; Openness focal area has made progress in promoting openness in research information through many internal discussions and wider projects. Our efforts have focused on supporting openness-driven data usage and decision making in science management and evaluation, and recognizing the important role open research information plays in these processes.

From vision to action: our first two years in the focal area Information &amp; Openness

Lutz Bornmann

For centuries, the scientific community has relied on visual tools to communicate (complex) data. But what if we could listen to data as effortlessly as we read a graph? This question lies at the heart of data sonification, a practice that translates datasets into sound.

Metrics sonification as a new way to convey bibliometric data

Thed van Leeuwen

Frank van Vree

(


 Klik hier om de Nederlandse versie te lezen


 )

In any evaluation of scientific research, the obvious question is: how does a research unit compare with others, nationally and internationally? This question is easier to ask than to answer, because the answer is not easy to substantiate.

Benchmarking in research evaluations: we can do without it

Blog team

Five years ago, we welcomed you to Leiden Madtrics through our first blog post. With the new blog, we wished to establish an active and engaging platform for our institute and our field to express ourselves – other than through scientific publications.

Celebrating 5 years of Leiden Madtrics

Alexander Schniedermann

Felicitas Heßelmann

Dimity Stephen

Max Leckert

Matteo Ottaviani

Since the inception of ChatGPT in late 2022, the question of how AI will change academia is a concern in almost every discipline. Social media and mailing lists alike are flooded with suggestions for new AI-based tools, workflows, or seminars. At the same time, many journal publications now come with some form of AI involvement – from analyzing and interpreting data to the drafting of whole sections.

Will ChatGPT Get Tenure?

Philippe Mongeon

Timothy D. Bowman

Leiden University

Rodrigo Costas

Wenceslao Arroyo-Machado

Introduction

On August 21, 2022, we made available the first version of our dataset of scholars on Twitter created with two open data sources: Crossref Event Data and OpenAlex.

An update on the Scholars on Twitter dataset

Nees Jan van Eck

Ludo Waltman

Zeynep Anli

Clara Calero-Medina

Dan Gibson

Mark Neijssel

Andrea Reyes Elizondo

Martijn Visser

Henri de Winter

Alfredo Yegros

Universities in the Leiden Ranking 2023

As Figure 1 shows, the number of universities in the Leiden Ranking keeps increasing. Like in the last three editions of the ranking, a university needs to have at least 800 fractionally counted publications in the most recent four-year time window to be included in the ranking. This year 1411 universities meet this criterion, 93 more than last year and 235 more than in 2020.

The CWTS Leiden Ranking 2023

Louise Bezuidenhout

Francesca Morselli

The transition towards open research practices has been a longstanding priority for Dutch academics. Over the last decade, the bottom-up support for open research practices has been accompanied by consolidated, national-level support. This country-level initiative has resulted in the 2017 National Plan for Open Science (NPOS) and the 2022 Open Science in 2030 in the Netherlands.

Monitoring Open Science in The Netherlands

Carlos Manuel Vilchez Román

Greetings from Peru, nestled in the heart of the Andes, where I find myself reflecting on two transformative events in my scientific journey: the CWTS Scientometrics Summer School (CS

 3

) and the 27th International Conference on Science, Technology, and Innovation Indicators (STI 2023). As a bibliometrics enthusiast for the past two decades, this experience has been nothing short of a revelation—a journey that prompted me to question

New Horizons: A reflection on the Scientometrics Summer School and the STI 2023 Conference

Dominique Babini

Arianna Becerril Garcia

Lautaro Matas

Ismael Rafols

Laura Rovelli

The Barcelona Declaration: a call for openness… but also for diversity and inclusion
 

The launch of the Barcelona Declaration last week aims to mobilise the global research community towards making research information open and accessible.

Not only Open, but also Diverse and Inclusive: Towards Decentralised and Federated Research Information Sources

Biegzat Mulati

Rong Ni

Jian Wang

Kwun Hang (Adrian) Lai

Marc Luwel

Ed Noyons

Verena Weimer

Open science was one of the key topics at the Science, Technology and Innovation Indicators (STI) conference that CWTS organised in September 2023 in Leiden, the Netherlands. Open science was not only discussed at the conference but was also put into practice in the publication and peer review process of the conference. By way of experiment, all papers submitted to the conference were published as a preprint before they were peer reviewed.

Preprinting and open peer review at the STI 2023 conference: Evaluation of an open science experiment

The release of the first Open Edition of the CWTS Leiden Ranking in January this year triggered an overwhelming response. It was covered by various news media, including Research Professional, AEF info, and TheMetaNews, and was endorsed by important stakeholders such as the International Science Council, the French Committee for Open Science, and the National Agency for Research and Development in Chile.

Launch of the CWTS Leiden Ranking Open Edition 2024

Jeremy Y. Ng

Henry Liu

Stefanie Haustein

Over the past decade, a growing number of bibliometric analyses of varying quality have been published in the peer-reviewed literature. Despite this growth, surprisingly few published articles provide guidance on how a bibliometric analysis ought to be reported. Moreover, to our knowledge, these articles have been written based on the opinions/experiences of different researchers, as opposed to best evidence-informed practices.

The GLOBAL initiative - Contribute to improving the reporting of bibliometric analyses

Bram van den Boomen

Tony Ross-Hellauer

Serge Horbach

The scientific community is moving towards a more transparent way of conducting and reporting research. Scientific publications are becoming more and more openly accessible but openness should also extend to peer review, preprinting, preregistration, data sharing, metadata availability, and related issues.Research funders and other stakeholders are putting a significant effort into promoting open science practices in scholarly communication.

The Journal Observatory - Connecting information on scholarly communication

A value-led perspective on Open Science
 

In 2021, UNESCO approved its Recommendation on Open Science (OS). By signing this recommendation, 193 countries made a commitment to support the development of OS with a vision of science as a global public good.

The UNESCO Open Science Outlook: OS progresses, but unequally

Tsuyoshi Hondou

Studies on transdisciplinary research often focus on how different forms of expertise are brought together to build robust knowledge. However, in policy and legal affairs, there are many situations in which it is not possible to use new transdisciplinary knowledge due to contextual factors, such as urgency, political expediency, or lack of resources.

Concurrent Evidence: a framework for using evidence from multiple disciplines

Sarah de Rijcke

In a previous blog post we introduced the CWTS knowledge agenda 2023-2028. Building on this, in the current post we present the new focal areas in which CWTS organizes its activities.

The focal areas of CWTS

To develop our CWTS knowledge agenda we formed three focal areas at the start of this year to organise our activities. Here, we introduce the vision and roadmap of the Focal Area Engagement &amp; Inclusion.

 Our vision

In today's rapidly changing world, fostering a collaborative, diverse and inclusive science ecosystem that engages with society is of paramount importance.

Introducing the CWTS Focal Area Engagement and Inclusion: A vision and roadmap

Academic research has become increasingly complex, multidisciplinary, collaborative, and transnational. The institutions that underpin research - including communication and evaluation systems - are trying to keep up, with varying levels of success. At the same time, our society is facing major challenges, including existential global health, welfare, and sustainability issues.

Introducing the CWTS knowledge agenda 2023-2028

Julián D. Cortés

Catalina Ramírez

Imagine national science policy as a musical chair game. The contestants are the science system actors, such as researchers, research groups, universities, companies, among others. Some actors can have more expertise dancing at the rhythm of

 salsa

than

 hip-hop,

while others might be more agile in finding a seat when the music pauses. The government plays or pauses the music, modulates its speed or changes the genre.

The changing tunes of science policy: mapping research priorities of consecutive governments

Ingeborg Meijer

The learnings have been presented and published in different shapes and forms, book chapters, working papers, reports, many deliverables, publications, and presentations. Here we present the breadth and depth of the work by linking it to the SUPER MoRRI blogpost series and turning them into an overarching story in two episodes.

A monitoring and evaluation framework for responsibility in R&amp;I

Tim Willemse

Sonia Mena Jara

Anestis Amanatidis

Tjitske Holtrop

Gaston Heimeriks

The EXPLORE methodology was developed in the context of the two territorial RRI projects CHERRIES and RIPEET (H2020) that CWTS is participating in since 2020. It can assist territories in developing more innovative, inclusive, and self-sustaining R&amp;I ecosystems by ensuring bottom-up involvement of all kinds of local stakeholders and citizens.

Cocreating science that society needs: introducing the EXPLORE approach

Giovanni Colavizza

The event

SciCon 2022 has been an online event seeking to bring together members of the academic metascience community and DeSci, to discuss some of the issues that exist in scientific research and highlight how DeSci efforts might contribute to improve the way we do science. SciCon has been organized by one of the leading DeSci initiatives: ResearchHub.

SciCon 2022: A look at Decentralized Science

Dmitry Kochetkov

Background: Russian Academic Excellence Project 5top100
 

The Russian Academic Excellence Project 5top100 is a Russian excellence initiative in higher education that ran from 2013 to 2020. The aim of the initiative was to place five Russian universities in the top 100 of global university rankings. The goal was not achieved;

What Affects Research Productivity: View from Inside the University

Magchiel Bijsterbosch

Alastair Dunning

Darco Jansen

Max Haring

Maurice Vanderfeesten

Research is increasingly data-driven, and so is the management, communication, and evaluation of research. The area of research intelligence is fuelled by big data analytics and provides new prospects for assisted decision-making on funding opportunities, publishing venues and next generation metrics.

Seven Guiding Principles for Open Research Information

Eugenio Petrovich

In spring 2020, Italy, like other countries in Europe and around the world, was in the midst of the first wave of the Covid-19 pandemic. Lockdown measures were in place in the entire country in the attempt to limit the spread of the virus.

Bibliometrics in press or the representation of indicators in the Italian news

Today CWTS published its open science policy on its website. This policy, which formally came into effect on September 1 this year, is the outcome of a co-creation process in which many CWTS colleagues participated in a period of one and a half years.

Practicing what we preach: Our journey toward open science

Leonie van Drooge

Previous posts on the Strategy Evaluation Protocol 2021-2027 describe the criteria and aspects and the process and responsibilities. Asking to judge research units on their own merits and putting the responsibility for the evaluation with the research organisations proves quite a challenge. Evaluation might not always happen according to protocol, yet the process itself offers opportunities to reflect and define. And as such it has value.

Research evaluation in context 4: the practice of research evaluation

In a previous post the SEP 2021-2027 was introduced. As a reminder: the Association of Universities in the Netherlands (VSNU), the Netherlands Organisation for Scientific Research (NWO) and the Royal Netherlands Academy of Arts and Sciences (KNAW) share the responsibility for the evaluation of all academic research units. According to the SEP 2021-2027, research units are evaluated in the light of their context, aims and strategy.

Research evaluation in context 3: The board, the research unit and the assessment committee

Laurens Hessels

Ad Prins

This is the last blog post on the Evaluative Inquiry, the new approach to research evaluation that CWTS has been developing since 2017, following one on
 
 broadening the concept of academic value
 
 ,
 
 
 
 evaluating research in context
 
 
 
 and
 
 
 
 mixing methods
 
 .

In evaluations of any kind people often distinguish between summative and formative

Evaluative Inquiry IV: Accountability and learning

‘Convergence’ as knowledge integration for grappling with societal challenges

Last October the US National Academies held a workshop (available here) to gather views on how to better measure and assess the implications of interdisciplinarity, or convergence, for research and innovation.

Knowledge integration for societal challenges: from interdisciplinarity to research portfolio analysis

Jennifer D'Souza

Search has long been revolutionized by knowledge-graph-powered services such as the Amazon Marketplace in e-commerce, or Open Street Maps in the cartography and navigation services domains, to name just two examples. Inspired by such knowledge graph (KG) success stories in the general domain, such technology is now being realized over scholarly knowledge as well.

Structuring Natural Language Processing Contributions in the Open Research Knowledge Graph

Juan Pablo Bascur Cifuentes

Why should you care?

How do you evaluate a university? This question does not have an easy answer. However, there is an attribute that is typically considered relevant: the production of scientific papers. However, papers are not created by the university

 per se

, they are created by

 individuals

who are affiliated to a university. Currently, how individuals contribute to the scientific output of a university (e.

Incorporating the human factor in the study of universities

Jonathan Dudek

François van Schalkwyk

Daniel Torres-Salinas

Nicolás Robinson-García

Tina Nane

One of the many puzzles of the COVID-19 pandemic has been the extent to which children are vulnerable to and spreaders of the virus. Fewer infection rates have been reported in children compared with adults, as have milder symptoms. But the infection rate in children is biased, given the testing policies in many countries.

Attention for science in step with policy? The case of school closures during COVID-19

Leo Tiokhin

This post is based on the following manuscript, currently under review: Tiokhin, L., Panchanathan, K., Lakens, D., Vazire, S., Morgan, T., &amp; Zollman, K. (Preprint). Honest signaling in academic publishing, and is also published in the author's blog
 
 author's blog).
 

“An article…is not the scholarship itself, it is merely advertising of the scholarship.” (Buckheit &amp;

Honest signaling in academic publishing

Interdisciplinary research for addressing societal problems

In this blog I will share some thoughts developed for and during a fantastic workshop (available here) held last October by the US National Academies to help the National Science Foundation (NSF) set an agenda on how to better measure and assess the implications of interdisciplinarity (or convergence) for research and innovation.

On 'measuring' interdisciplinarity: from indicators to indicating

Cassidy R. Sugimoto

Vincent Larivière

In this blog post we discuss recent findings on the relation between career trajectories and task specialization. Research careers are commonly envisioned in evaluation schemes as homogeneous pathways in which individuals have to take a series of steps to advance.

The unintended consequences of task specialization in research careers

English version below


 .

Vorige week werd in de Rijksbegroting een halvering van het budget van Open Science NL aangekondigd. Open Science NL, een zogeheten regieorgaan van NWO, heeft als opdracht om de transitie naar open science in Nederland te stimuleren en versnellen, met als doel dat open science de norm wordt.

Bezuiniging op Open Science NL, superzonde!

Peter van Dongen

Robert Tijssen

Excellent science
 

Achieving scientific success depends on several factors: inspiration, creativity, hard work, access to the most advance instruments, et cetera.

The X factor of eXcellent scientists in the Netherlands: relationships between motivation, utilisation and impact

Lidia Carballo-Costa

The Gender Inequalities in Science workshop took place at Leiden University on 7th-8th October 2019, organized jointly by CWTS and Elsevier’s International Center for the Study of Research (ICSR). For two days, researchers from different research institutions, universities and science stakeholders discussed issues concerning gender in science from diverse perspectives and contexts.

Gender inequalities in science: Evidence and ideas from bibliometrics

Tim Wölfle

I'm sure the following scenario is familiar to most of the readers: You're writing a new scientific paper and are scanning the reference lists of your most important sources to see if you've missed anything interesting, maybe even a seminal paper cited by several of your current sources. Recently, a couple of web apps have come up to simplify this process by constructing and visualizing the underlying citation networks.

Local Citation Network and Citation Gecko: making literature discovery fun

This blogpost is a follow-up of the post by Carole de Bordes. She discusses a pre-covid19 workshop she attended on virtual meetings and conferences. Due to the Covid-19 crisis, we are currently experiencing a new way of working at CWTS. All staff needs to work from home. And besides the difficulties some colleagues are facing in this transition because of their home situations, most of the normal work could be taken up quite fast again.

Going back to normal?

One of the most exciting developments in the past few years in the field of bibliometrics is the emergence of a number of important new data sources. Dimensions, created by Digital Science and made openly available for research purposes, is a prominent example. Other examples are Crossref and OpenCitations, which provide data that is fully open. The launch of Microsoft Academic in 2016 also represents a significant development.

Mapping science using Microsoft Academic data

Wytske Hepkema

Willem Halffman

What is PREP?

PREP is an online platform contributing to the responsible organisation of editorial procedures by scholarly journals. It facilitates journal editors to become transparent about their editorial procedures, advises journal editors and publishers on potential improvements of their peer review procedures, and presents integrated information about the variety of review procedures currently in use.

Launch of Platform for Responsible Editorial Policies

Rinze Benedictus

“Fund and forget”, that is how Jordi Molas-Gallart from the research centre Ingenio (CSIC-UPV) in València teasingly characterized the way large national research funders often operate. It’s also a point of departure for thinking about new ways of research evaluation. The cornerstones of many forms of research evaluation are peer review and bibliometric analysis.

From ‘fund and forget’ to formative and participatory research evaluation

Your paper is based on a huge amount of data. How did you manage to get access to so much data?

In the internal database system of CWTS, we have access to the raw data of Scopus, Web of Science, Dimensions, Crossref, and Microsoft Academic. CWTS is probably the only center in the world that has access to all this data, so we are in a unique position to compare the different data sources.

Comparing bibliographic data sources: Q&amp;A

Today CWTS is releasing the 2020 edition of the CWTS Leiden Ranking. As always, the release of the ranking requires a major effort involving contributions from a large number of colleagues at our center. This year, the COVID-19 crisis, combined with a major update of the internal data infrastructure at CWTS, created additional challenges, which has unfortunately led to a delay of almost two months in the release of the ranking.

The CWTS Leiden Ranking 2020

Zhichao Fang

In response to the COVID-19 pandemic, researchers from all over the world and from multidisciplinary backgrounds are working on answers to the challenges raised by the disease. In this regard, the biomedical point of view is also accompanied by broader social, economic and political perspectives.

Monitoring the dissemination of COVID-19-related scientific publications in online media

Sarah Rose Bieszczad

Jochem Zuijderwijk

Months into the corona crisis we are living in a world changed, for better or worse things are

 other

than they once were. It is obvious many countries were underprepared, but does it make sense to ask—could this have been otherwise? Indeed, this is a prodigious question, one unlikely to be answered comprehensively by a blog post.

Could pre-Covid-19 research into coronaviruses have been otherwise? Episode one: Careers

Any conversation about research evaluation in the Netherlands has the risk of developing along this line. The Dutch way of evaluating academic research might not be unique, but it is certainly not common, nor fully understood. As a member of the working group for the monitoring and further development of the evaluation protocol – and as an employee of CWTS – let me provide insight and context.

Research evaluation in context 1: Introducing research evaluation in the Netherlands

Bastian Drees

Grischa Fraumann

Isabella Peters

This article has been originally published at the TIB Blog.



At the TIB AV-Portal, we have created a service to watch the Altmetrics Conferences and Workshops.

You couldn't attend the Altmetrics conference? Fear not! We recorded everything you missed

Clare Shelley-Egan

Rune Nydal

Mads Dahl Gjefsen

RRI &amp; OS: two co-existing sets of ambitions

Responsible Research and Innovation (RRI) and Open Science (OS) are two co-existing sets of ambitions concerning systemic change in the research and innovation (R&amp;I) system. Initially, RRI and OS may appear to align well.

Responsible Research &amp; Innovation or Open Science - does the label matter?

Bianca Kramer

Why is it important that abstracts are made openly available?

In a blog post that we published two years ago together with our colleague Aaron Tay, we discussed numerous ways in which open abstracts can be of great value. This includes scientific literature search, both using traditional query-based search tools and using more advanced text mining and visualization tools.

Q&amp;A about Wiley's decision to open its abstracts

The year is almost over. 2022 was not just another year for the Leiden Madtrics blog, but came with quite a variety of blog posts. We (the blog team) had a lot of fun editing all of them and are proud of our authors. They did an amazing job.  At the same time, this year diversified our social media presence: Leiden Madtrics, along with CWTS, can now be found on Mastodon as well, while the institute also launched its own Mastodon instance.

Leiden Madtrics

The Pain of Labeling Things

Mapping science using Microsoft Academic data

Why are global health needs unmet by research efforts?