Comment & practical guidance from the LIS-Bibliometrics community

Bibliomagician Staff

On Wednesday, 15

 th

June I had the pleasure of attending a Jisc-supported Thomson-Reuters one-day event at Edinburgh University Library on the theory, and practice of bibliometric analysis for evaluating research and planning research policy.

Event report: Theory &amp; Practice of Bibliometric Analysis by Charles Oppenheim

In light of rumours that only DORA signatories will have access to UKRI funding in future, Lizzie Gadd explains why Loughborough has chosen an alternative path up the ‘responsible metrics mountain’, and why she believes all ‘mountain-climbers’ should be equally supported and rewarded.

As one of a small, but seemingly growing, group of UK universities who have given serious thought to signing the San Francisco Declaration on Research

DORA, the Leiden Manifesto &amp; a university’s right to choose

A few months ago I asked the lis-bibliometrics list how institutions that had purchased SciVal had rolled it out across their respective universities.  We have recently taken out a subscription to SciVal at my own institution (De Montfort University in Leicester, UK) and we are starting to think about the services and support we want to provide.

Summary of different institutions’ approach to rolling out SciVal and support services. Guest post by Jenny Coombs.

In this post, distinguished Professor Henk Moed explains why he has published his recent book
 
 Applied Evaluative Informetrics
 
 , and outlines his critical views on a series of fundamental problems in the current use of research performance indicators in research assessment.

How can we use research performance indicators in an informed and responsible manner? Guest post by Henk Moed

Lizzie Gadd reflects on two recent responsible metrics events: Lis-Bibliometrics’ “Responsible Bibliometrics in Practice” and the HEFCE/UUK’s “Turning the Tide: Changing the Culture of Responsible Metrics” both held within ten days of each other in London.

Responsible metrics events are like buses it would seem.  Nothing for nine months, and then three within the space of ten days.

Responsible metrics: where it’s at?

Here are the links to presentations given at the recent

 #

respbib18 conference at Altmetric, 30 January 2018. Many thanks to all the presenters, and to the participants, for a stimulating day. 9:00-9:45

 Registration and refreshments

9:45-9:55

Outputs: Responsible Bibliometrics in Practice Event

Dr Quentin Pope, a Liaison Librarian and subject specialised librarian with specific responsibilities for the library and information requirements of QUT School of Law, outlines how you can track citations in Law.

The primary purpose of bibliographic reference research is to link references in citations of scholarly publications.

Tracking citations in Law – Guest Post by Quentin Pope

Ian Rowlands hammers another nail into the h-index’s coffin by explaining how it actually brings nothing new to the table and by showing how chance determines its outcome.

The poor old h-index has come in for a lot of criticism recently, with a flood of papers pointing to its conceptual and practical shortcomings.  Is it time to bury it?

Is it time to bury the h-index?

Lizzie Gadd spent some time with the Research Intelligence Network of the Netherlands, and explains why she’s a little bit in love with the Dutch…

A couple of weeks ago I was lucky enough to spend a few days in Amsterdam talking to research evaluation colleagues from across the Netherlands.

Research evaluation: things we can learn from the Dutch

Applications to join the LIS Bibliometrics Committee are now open!

We are looking for enthusiastic and committed individuals to join the LIS Bibliometrics Committee in the below positions.

Join the Committee

In this post, Stacy Konkiel (Director of Research Relations at


 Altmetric, a Digital Science company)


 summaries feedback of a recently posed question


 in identifying authors’ genders based upon names


 listed on ~2,000 journal articles–too large a corpus for manual analysis.

Recently on the LIS-Bibliometrics listserv, Ruth Harrison (Imperial College London) posed a question on behalf of a patron who

A guide to identifying author gender for bibliometric analyses – Guest post by Stacy Konkiel

On 26th March 2018, the Lis-Bibliometrics Forum launched a practical course: Statistics for Responsible Metrics aimed to offer a bespoke training that focused exclusively on statistics for bibliometrics.

Course summary: Statistics for Responsible Bibliometrics

In July the UK’s four higher education funding bodies published the draft submission guidance and panel criteria for the Research Excellence Framework (REF) 2021, for consultation before finalising them. Each makes mention of the use of bibliometrics, so I’d recommend reading them to all those involved in the intersection of REF and bibliometrics. As well as these two consultations, there is also a guidance document on environmental indicators.

Metrics in latest REF documents

Lizzie Gadd and Ian Rowlands report on the Lis-Bibliometrics survey of end-users around how bibliometric and altmetric suppliers might improve their services.

It’s fun being on the Lis-Bibliometrics Committee.

Three ways metrics vendors can improve – messages from end-users

Lizzie Gadd considers the practical implications of the responsible research evaluation requirement of the Wellcome Trust’s new Open Access Policy.

Responsible research evaluation is important. Really important. It’s not only quality-of-life important but life-or-death important to many over-worked, over-evaluated and under-resourced researchers.

Mandating responsible research evaluation: the way to go?

In this guest post, Jeffrey Demaine explores some tipping points in Open scholarly communication, and what it means in the context of Canada’s research landscape.

After more than a decade of incremental progress towards OpenAccess publishing, some national research-funding bodies are ready to engineer disruptive change in the way scholarly communication works.

Citation issues resulting from Canada’s new RDM mandate – a tipping point for Open Science? Guest post by Jeffrey Demaine

Lizzie Gadd makes the case for open research being required not rewarded.

I recently attended two events: the first was a workshop run by the ON-MERRIT team, a Horizon 2020 project seeking to understand how open research practices might actually worsen existing inequalities. And the second was the UKRI Enhancing Research Culture event at which I was invited to sit on a panel discussing how to foster an open research culture.

How (not) to incentivise open research

Jonathan Adams from Clarivate Analytics discusses how to responsibly use and interpret bibliometric data through a list of caveats and requirements prior to starting the planning of research evaluations.

Ensuring the responsible use of data about research activity has become an increasing problem over the last twenty years.

Guest post: Responsible research evaluation starts here

Welcome to the first Altmetrics Research Roundup! In these regular posts, I’ll be sharing and summarizing the altmetrics research that’s most relevant to practicing bibliometricians’ daily work.

Tips for reading and interpreting altmetrics research (Guest post by Stacy Konkiel)

Stephanie Meece, Scholarly Communications Manager at University of the Arts London, summarised the Bibliometrics in the Arts, Humanities and Social Sciences conference at the end of the event. Here she offers her reflections on the event.
 

On Friday, 24 March 2017, I attended the meeting of the LIS-Bibliometrics group in London.

A review of the #AHSSmetrics conference

The London School of Economics and Political Science (LSE) is currently advertising for a

 Research Information Analyst &amp;

Job Opportunity: Research Information Analyst &amp; Open Access Officer, LSE

Leicester is recruiting a permanent, full-time bibliometrician and explains the role thus:  You will provide high-level expertise, and advice to the University on the use of bibliometrics and related indicators in support of the University’s research objectives and to ensure that the University maintains a comprehensive record of its research outputs and that these are correctly indexed by the major citation services.

Job Opportunity: Bibliometrician and Data Analyst at University of Leicester

Rachel Miles, Research Impact Librarian at Virginia Polytechnic Institute and State University, summaries the results from a recent National Survey, giving an insight on how Academic Librarians are using altmetrics in the US.


 Altmetrics: What is going on?


 “There is a growing interest in altmetrics.”


 “Altmetrics have become the newest tool among academic librarians.”


 “Scholarly communication is

How Do Academic Librarians Use Research Impact Metrics? Guest post by Rachel Miles

Chalmers University of Technology in Sweden is looking for a bibliometric analyst.

Chalmers, a highly progressive technological university, is situated on the west coast of Sweden in beautiful Gothenburg.

 Information about the research/the project/the department



Chalmers Library is expanding its work with bibliometrics and university ranking analytics.

Job Opportunity: Bibliometric Analyst at Chalmers University of Technology

The four UK higher education funding bodies are consulting on proposals for the next Research Excellence Framework.  Thank you to all Lis-Bibliometrics members who have contributed their thoughts on this.  Here is a draft response the Lis-Bibliometrics Committee intends to submit on behalf of the group.  If you have any last minute comments please contact me or share via the list as soon as possible.

REF consultation: Lis-Bibliometrics response

Here are the links to presentations given at the recent #AHSSmetrics conference at the University of Westminster, 24 March 2017. Many thanks to all the presenters, and to the participants, for a stimulating day. For those who missed the event, Karen Rowlett has helpfully created a Storify of the tweets at https://storify.com/karenanya/bibliometrics-for-the-arts-and-humanities.

Outputs from Bibliometrics in Arts, Humanities and Social Sciences conference

JUNE 2016 WILL GO DOWN IN THE ANNALS OF HISTORY FOR A NUMBER OF REASONS…Britain voted to leave the European Union, Andy Murray won Wimbledon for the second time, it was the hottest month ever recorded in history …and

 “Bibliometrics in Practice”

took place in Manchester!

Round up from Bibliometrics in Practice event

By Nicolas Robinson Garcia and Lizzie Gadd Since 2015, the LIS-Bibliometrics Committee has run annual Responsible Metrics State-of-the-Art surveys in which we analyse the penetration of responsible metrics awareness and implementation in universities. Every year, the number of participants as well as the number of countries involved has increased, and already last year reported a total of 115 respondents.

Results from the 2019 Responsible Metrics State of the Art Survey

So, I’ve just returned from the excellent JISC/CNI conference on advances in digital scholarship where they had a panel session on metrics.

Is signing DORA that responsible?

When I started working with bibliometrics I was aware of the limitations and criticisms of journal metrics (journal impact factor, SJR, SNIP etc.) so I avoided using them.  A couple of years on, I haven’t changed my mind about any of those limitations and yet I use journal metrics regularly.  Is this justifiable?  Is there a better alternative?  At the moment, I’m thinking in terms of filters.

In search of better filters than journal metrics

We are currently advertising for a permanent full time Copyright and Scholarly Communications Manager at the University of Salford. The University Library is looking to appoint a highly motivated individual to the position of Copyright and Scholarly Communications Manager. You will have responsibility for the management and development of our copyright service and the copyright licence for the University of Salford.

Job Opportunity: Copyright and Scholarly Communications Manager, University of Salford

Dr Ian Rowlands, writing in a personal capacity   Let’s begin with a self-evident truth: publication counts aside, bibliometric indicators often change each time new material is added to a citation database.  That’s a given, but it begs a couple of interesting questions: how much do indicators change, and how quickly?

Six weeks is a long time in bibliometrics: Stability and Field-Weighted Citation Percentile

University of Grenwich is recruting a new Scholarly Communications Manager.

Greenwich Research &amp; Enterprise is the University’s central office responsible for developing a supportive research culture and establishing links with industry and enterprise. GRE works across four service areas: research development services, commercialisation and innovation services, research &amp; enterprise training (including postgraduate administration and training) and business support services.

Job Opportunity: The University of Greenwich is recruiting a new Scholarly Communications Manager

Josh Nicholson writes about how to use scite, a platform for discovering and evaluating scientific articles via Smart Citations.

scite: more than just a citation index

Rachel Miles, Research Impact Librarian, and Amanda MacDonald, Undergraduate Research Services Librarian, discuss how to use altmetrics as a critical evaluation tool in undergraduate and graduate instruction to trace mentions of research and delve into the context and constructs of the mentions surrounding it.

Alternative metrics (altmetrics) measure the volume of online attention to research, which can come from a variety of

Altmetrics as an Information Evaluation Tool

Lizzie Gadd gets all fancy talking about algorithms, machine learning and artificial intelligence. And how tools using these technologies to make evaluative judgements about publications are making her nervous.

A couple of weeks ago, The Bibliomagician posted an interesting piece by Josh Nicholson introducing scite.

AI-based citation evaluation tools: good, bad or ugly?

Lizzie Gadd speculates as to why Elsevier endorsed the Leiden Manifesto rather than signing DORA, and what the implications might be.

If an organisation wants to make a public commitment to responsible research evaluation they have three main options: i) sign DORA, ii) endorse the Leiden Manifesto (LM), or iii) go bespoke – usually with a statement based on DORA, the LM, or the Metric Tide principles.

Elsevier have endorsed the Leiden Manifesto: so what?

Librarians Fiona Glasgow and Dr Antje Lübcke discuss how they trained their colleagues in bibliometrics with fun and creative workshops at the University of Otago, NZ. The 11 IATUL Research Impact Things&#160; How do you make bibliometrics fun? That’s the question we set out to answer when our manager told us we had to roll &#8230; <a href="https://thebibliomagician.wordpress.com/2021/01/28/how-to-make-bibliometrics-fun/" class="more-link">Continue reading "Guest post: How do you make bibliometrics fun? Training librarians at the University of Otago,&#160;NZ"</a>

Librarians Fiona Glasgow and
 
 Dr Antje Lübcke
 


 discuss how they trained their colleagues in bibliometrics with fun and creative workshops
 
 at the University of Otago, NZ
 

. The 11 IATUL Research Impact Things  How do you make bibliometrics fun?

Guest post: How do you make bibliometrics fun? Training librarians at the University of Otago, NZ

Lizzie Gadd &amp; Gareth Cole discuss the practical challenges of monitoring progress towards institutional open research data ambitions

. Loughborough University has recently introduced a new Open Research Position Statement which sets out some clear ambitions for open access, open data and open methods. As part of this work we’re looking at how we can monitor our progress against those ambitions.

The challenge of measuring open research data

Part two of a three-part blog summary of the discussion at the ‘Implementing responsible research assessment’ panel at the LIS-Bibliometrics 10th anniversary event: The Future of Research Evaluation in March 2020. Some of the questions put to the panel and their responses have been collected and grouped by theme.

Guest Post: Implementing responsible research assessment – governance and rankings

Marianne Gauffriau (Copenhagen University Library, Denmark) and Yrjö Leino (CSC – IT Center for Science, Finland) explore SciVal’s calculation of field-weighted percentile indicators and the lesser-known implications of this method.

Introduction This blog post is inspired by a question frequently posed to Marianne in her role as Coordinator of the Bibliometric Service at Copenhagen University Library, Denmark.

Guest Post: Understanding SciVal’s calculation of field-weighted percentile indicators

Hannelore Vanhaverbeke, Barbara Lancho Barrantes &#38; Andrew Cox are set to update the “Competency model for bibliometrics work” and after gathering feedback they present their early findings here. A Time for Review In view of the fast-paced changes in bibliometrics, both in terms of new indicators, the ‘next-generation metrics,’ and in terms of new technology, &#8230; <a href="https://thebibliomagician.wordpress.com/2021/03/15/future-competencies-in-bibliometrics/" class="more-link">Continue reading "“Don’t stop thinking about tomorrow!” – Future Competencies in&#160;Bibliometrics"</a>

Hannelore Vanhaverbeke, Barbara Lancho Barrantes &amp;

“Don’t stop thinking about tomorrow!” – Future Competencies in Bibliometrics

An update and announcement about the (virtual) 2021 LIS-Bibliometrics Conference.


 Update:

registration now open! Check out the LIS-B Conference website for details.

LIS-Bibliometrics Conference 2021: “New tools and new technologies: bibliometrics in action for a new decade”

Luciana Balboa, Elizabeth Gadd, Eva Mendez, Janne Pölönen, Karen Stroobants, Erzsebet Toth Cithra and the CoARA Steering Board


 respond to a blog post by scientometrician, Giovanni Abramo, addressing specific concerns the scientometric community has with the Coalition for Advancing Research Assessment (CoARA), specifically The Agreement on Reforming Research Assessment.


 Introduction

The role of scientometrics in the pursuit of responsible research assessment

The theme for the LIS-Bibliometrics conference 2024 is
 
 Exploring the Bibliometric Universe
 
 . LIS-Bibliometrics conference 2024 wishes to explore the ways that Bibliometrics is being used in the wider academic universe and help those who use bibliometric tools navigate that universe.


 The three keynote speakers are Professor Jennifer Byrne, Dr. Álvaro Cabezas Clavijo

,

 and Professor Caroline S. Wagner.

Announcing the Keynote Speakers for the 2024 LIS-Bibliometrics Conference

Cecile Janssens, co-developer of the citation-based search tool CoCites, discusses how such a tool can rapidly and accurately identify relevant scientific articles based on citation and co-citation frequency. Janssens also elaborates on how CoCites can improve its efficiency, intuitiveness, accountability, and reproducibility.

Citation-based search tools: the sky is the limit, but what do we need?

LIS-Bibliometrics Committee members


 Barbara S. Lancho Barrantes, Hannelore Vanhaverbeke and Silvia Dobre


 discuss the new bibliometrics competencies model, why it was updated, and the changes made to it.

The “Competencies for bibliometrics” paper was first published online in the

 Journal of Librarianship and Information Science

in 2017 and informed the original Competencies Model for Bibliometric Work,

The New Competencies Model for Bibliometrics

Today the CWTS Leiden Ranking and the INORMS More Than Our Rank initiative announce a new partnership, aimed at highlighting the accomplishments of universities beyond what is captured in university rankings.


 This blog post originally appeared on Leiden Madtrics and has been reposted here for redistribution and sharing purposes.

The CWTS Leiden Ranking has always been a bit different from other university rankings.

The Leiden Ranking goes beyond ranking

Rachel Miles summarizes a recently published journal article that cautions researchers on the use of automated citation recommendation tools, which are designed to help academic authors cite literature in the writing phase of their research project while potentially skipping crucial steps of searching, reviewing, and appraising literature.

In an increasingly competitive academic environment, researchers are understandably keen to find

Use with caution! How automated citation recommendation tools may distort science

Robyn Price examines the data and methods of the PLOS Open Science Indicators dataset to reflect on the questions that institutions might need to ask themselves before looking to expand open research metrics As research organisations, funders and publishers pursue open research agendas amidst a broader ‘datafication’ of the sector, interest will likely only grow &#8230; <a href="https://thebibliomagician.wordpress.com/2023/02/16/preparing-for-open-science-indicators/" class="more-link">Continue reading "Are research organisations ready for open science&#160;indicators?"</a>

Robyn Price
 
 
 examines the data and methods of the PLOS Open Science Indicators dataset to reflect on the questions that institutions might need to ask themselves before looking to expand open research metrics

As research organisations, funders and publishers pursue open research agendas amidst a broader ‘datafication’ of the sector, interest will likely only grow for metrics that go beyond open access status or

Are research organisations ready for open science indicators?

Dennis Donathan, a Research Assistant and Data Scientist for the Public Knowledge Project, discusses a recent study he conducted on Reddit users’ engagement with research outputs. His research showed that although Reddit is not typically the focus of altmetrics research, the platform can act as an informative and valuable tool in altmetrics research.

Reddit: an important tool in the research engagement tool bag

Our 2022 Conference took place online on 5 and 12 October. If you missed it or want to re-watch a session you can do so on our Zenodo channel: https://zenodo.org/communities/lis-bibliometrics/  ‘Measuring What Matters’  Our conference theme aimed to explore responsible and values-driven uses of metric indicators in research assessment.

2022 LIS-Bibliometrics Wrap Up

Liam Bullingham and Nicola Wylie report on an event focusing on the individual signers of DORA. Let’s focus on people Recent discussion on DORA mostly concerns the 2,800 organisations (e.g. universities, funders or publishers) which have signed, not the 20,000 individuals. See positions like, ‘Why have Elsevier committed to Leiden but not DORA?’, ‘We as &#8230; <a href="https://thebibliomagician.wordpress.com/2023/07/06/why-i-signed-dora/" class="more-link">Continue reading "‘Why I signed&#160;DORA’"</a>

Liam Bullingham
 


 and


 
 Nicola Wylie
 


 report on an event focusing on the individual signers of DORA.

Let’s focus on people Recent discussion on DORA mostly concerns the 2,800 organisations (e.g. universities, funders or publishers) which have signed, not the 20,000 individuals.

‘Why I signed DORA’

J

 effrey Demaine a Canadian Bibliomatrician reflects on the
 
 28th Nordic Workshop on Bibliometrics and Research Policy
 
 in Gothenburg, Sweden, providing a thoughtful summary of the state of the field in the EU.

In October 2023, I attended the

 28th Nordic Workshop on Bibliometrics and Research Policy

in Gothenburg, Sweden.

Reflections on the 28th Nordic Workshop on Bibliometrics

Scientometric researcher and President of the International Society for Scientometrics and Informetrics (ISSI), Dr. Giovanni Abramo
 
 , discusses his views on and critically analyzes the Coalition for Advancing Research Assessment (CoARA) Initiative. A response is forthcoming from CoARA.

The Battle of Peer Review and Scientometric Assessment: Analyzing the CoARA Initiative

Authors Maddie Hare, Heather Woods, Isabelle Dorsch, Carey Ming-Li Chen, and Stefanie Haustein, discuss their groundbreaking work on the Metrics Literacies Project.

Developing Metrics Literacies: Competencies, dispositions, and knowledge for the critical assessment and ethical use of scholarly metrics

We are delighted to share some news about our next conference with the bibliometrics community. While we were initially planning to hold our annual conference this year, we’ve decided to take a bold leap and make it even better by hosting it

 both online and in person

next year.

Exciting News: The Lis-Bibliometrics Conference is going hybrid in 2024!

An interview with Juan Gorraiz, director of the Bibliometrics and Publications Strategies department at Vienna University Library

Nicolas Robinson interviews Juan Gorraiz, who works at Vienna University Library as a professional bibliometrician. Juan is editing a special issue for Frontiers in Research Metrics and Analytics on best practices in bibliometrics.

An interview with Juan Gorraiz

Robyn Price considers how systematic injustices in academia are present in and perpetuate in bibliometrics; and ways that the bibliometric community can address this.

Working in bibliometrics is difficult. The part that I, and I think a lot of people in this community, find difficult is the responsibility for data about research outputs and the people who produce research.

Truth behind the numbers?

Lizzie Gadd considers whether “responsible” is something we do or something we are.

I’ve made the point a few times recently that responsible bibliometrics requires responsible bibliometricians.  It is not just our systems and processes that need to be responsible, but us.  We are the ones supporting these activities in our institutions.

What does a responsible bibliometrician look like?

At Loughborough University we have recently been thinking about how we can use bibliometrics responsibly.  Not surprisingly our conversations tended to focus on journal and conference papers where the majority of citation databases focus.  However, as part of this process the question inevitably arose as to whether there were also ways we could measure the quality or visibility of non-journal outputs, in particular, monographs.

Measuring the magnificence of monographs

On Wednesday 8

 th

February 2017, Imperial College made headlines by announcing that it has signed the San Francisco Declaration on Research Assessment. Meaning that Imperial will no longer consider journal-based metrics, such as journal impact factors, in decisions on the hiring and promotion of academic staff.

HEFCE: The road to the Responsible Research Metrics Forum – Guest post by Ben Johnson

In September 2015 I ran a short survey to see how institutions were responding to the Metric Tide report‘s call to take a responsible approach to metrics and to consider signing the San Francisco Declaration on Research Assessment (DORA). Only three of the survey respondents had signed DORA (although 5 were considering it), but nine respondents were thinking about developing their own set of principles for the responsible use of metrics.

UK universities respond to the Metric Tide’s call to do metrics responsibly

Kick starting the series of “In a day of a Bibliomagician” we focus on what it is like to be a Bibliometrician in Canada. Shannon Gordon who is a

 
 Bibliometrics and Reseach Impact Librarian
 

at the University of Waterloo, shares with us her experience. Here is her interview.

 Shannon, t


 hank you very much


 for agreeing to be the first interviewee for this series.

In a day of a Bibliomagician (Shannon Gordon)

Dr Alice Gibson considers how developments in open access publishing, including transformative agreements, have added to the complexity in equitable practices surrounding credit for authorship.

Author matters Following the launch of Plan S in 2018, there has been increasing awareness and discussion of the meaning and status of the role of a corresponding author.

New Open Access Models and The Allocation of Author Roles: Can We Embed Equity Within an Unfair System?

Robyn Price from Imperial College London briefs us on details of the THE Impact Ranking methodology

The fourth edition of The Times Higher Education (THE) Impact Ranking is set to publish in April 2022 and will surely be followed by participating institutions communicating successful performance in it and non-participating institutions might be considering whether to take part in future editions.

Brace for impact: thoughts on the 2022 THE Impact Ranking

Having worked in UK academic libraries for 15 years before becoming freelance, I saw the rise and rise of citation counting (although as Geoffrey Bilder points out, it should rightly be called reference counting). Such counting, I learnt, was called “bibliometrics”. The very name sounds like something that librarians should be interested in if not expert at, and so I delved into what they were and how they might help me and also the users of

A useful tool for librarians: metrics knowledge in bite-sized pieces By Jenny Delasalle

“Measuring what matters”

 Call for conference committee members

This year’s LIS-Bibliometrics Conference on the theme of “Measuring what matters” will be held

 online

in

 September 2022.

We are seeking volunteers from across and beyond the HE sector to join the organising committee.

LIS-Bibliometrics Conference 2022

David Whyte from the University of Liverpool thinks his institution is making a mockery of established standards in research metrics. In this post he breaks down why he thinks so and how this could effect the broader research community.

I’ll begin by putting my cards on the table.  This blog comes from a very particular perspective.

Dressing up Irresponsible Metrics as Responsible Metrics

Dr. Nicolas Robinson-Garcia discusses the results of the 2020 Responsible Metrics State of the Art Survey, which has been conducted every year since 2015.

With delay we are happy to report the findings of the 2020 Responsible Metrics State of the Art Survey. Being 2020 a difficult year for everyone, we do observe a decrease in participation with respect to 2019’s results going from 218 respondents to a total of 139 respondents.

Results from the 2020 Responsible Metrics State of the Art Survey

Part one of a three-part blog series summarizing the discussion of the ‘Implementing responsible research assessment’ panel, at the LIS-Bibliometrics 10th anniversary event: The Future of Research Evaluation in March 2020. Some of the questions put to the panel and their responses have been collected and grouped by theme.

Guest Post: Implementing responsible research assessment – challenges of implementation

Lizzie Gadd encourages us to clarify what, why and how we are measuring before launching into any conversation about responsible metrics.

I love the poem, “Five Blind Men and an Elephant” by John Godfrey Saxe.

The blind and the elephant: bringing clarity to our conversations about responsible metrics

Lizzie Gadd asks whether more could be done to build confidence in the REF peer review process.


 Introduction

Peer review is still held up as the gold standard of research evaluation, and, for most, infinitely preferable to metrics. As such, the REF prides itself on being “a process of expert peer review”, so reads its guiding principle. But the status of peer review has taken rather a battering in recent years.

Three thoughts about REF peer review

Laura Himanen and Lizzie Gadd introduce a step-by-step process developed by the INORMS Research Evaluation Working Group for evaluating responsibly.

We’ve got a shed load of principles now for responsible research metrics. We have DORA, the Leiden Manifesto and the Metric Tide. We also have the many bespoke sets of principles being developed by individual organisations. And they’re great.

Introducing SCOPE – a	process for evaluating	 responsibly

Scientometric researchers, Drs. Giovanni Abramo and Andrea D’Angelo, discuss their recent study on Italy’s national accreditation system (ASN) and whether it incentivizes specialization or diversification of researchers’ portfolios. The results may surprise you!

Imagine a world where scientists and researchers have to balance their passion for advancing knowledge with an ever-watchful eye on performance metrics.

Do research assessment systems have the potential to hinder scientists from diversifying their research pursuits?

Rachel Miles, lead author of the guide, discusses some key points from


 the new Using Altmetric Data Responsibly guide


 , which is now added to the collection of


 Responsible Use Guides


 from the LIS-Bibliometrics Community.

Introducing the new responsible use guide: Using Altmetric Data Responsibly: A Guide to Interpretation and Best Practice

The results of the seventh annual Responsible Metrics State of the Art survey are in! With a little delay on our usual schedule of publishing, please find below the summary of the results of both the 2021 survey and some reflection on the trends since the survey was first begun by the LIS Bibliometrics community in 2015. Prior year survey results are also available. Response demographics  A total of 91 responses to the 2021 survey were received.

Results from the 2021 Responsible Metrics State of the Art Survey

LIS-Bibliometrics are delighted to announce the appointment of Robyn Price and Zosia Beckles as the new Co-Chairs of the LIS-Bibliometrics Committee. They bring with them a wide range of analytics experience across the UK HE and research sectors.

Announcement: New LIS-Bibliometrics Committee Co-Chairs

2021’s LIS B conference, like many things this year, required some changes. As we moved the conference online, we had to figure out which platform would be the easiest to use for presenters and attendees, how best to accommodate presenters in multiple time zones, and what we would do if all of the more than 300 registrants actually showed up on each day.

2021 LIS-Bibliometrics Conference Wrap Up

Part three of a three-part blog summary of the ‘Implementing responsible research assessment’ panel at the LIS-Bibliometrics 10th anniversary event: The Future of Research Evaluation in March 2020. Some of the questions put to the panel and their responses have been collected and grouped by theme. The panel comprised of Sarah Slowe (University of Kent), Steven Vidovic (University of Southampton) and Karen Desborough (Cardiff University).

Guest Post: Implementing responsible research assessment – monitoring, compliance and beyond DORA

SciVal’s field-weighted citation impact (FWCI) is an article-level metric that takes the form of a simple ratio: actual citations to a given output divided by the expected rate for outputs of similar age, subject and publication type.  FWCI has the dual merits of simplicity and ease of interpretation: a value of 2 indicates that an output has achieved twice the expected impact relative to the world literature.

SciVal’s Field weighted citation impact: Sample size matters!

In the world of academia and research, “publish or perish” has become more complicated than ever. It’s not enough to merely publish, one has to publish in a high-impact journal, in the hopes of getting noticed, and more importantly, perhaps, getting funded for further research. Institutions are urging their researchers to publish in high-impact journals.

Journal Metrics in Context: How Should a Bibliomagician Evaluate a Journal? Guest post by Massimo Giunta

There are many stakeholders in the world of research evaluation by numbers, and the Metric Tide Report did a great job of teasing out who should be doing what. At the ARMA Conference this year, the Metrics Special Interest Group (SIG) brought together a fantastic panel of those stakeholders to get an understanding of how they have responded to the Metric Tide report: what is happening in the world of responsible metrics, and what needs to happen next?

ARMA Metrics Champion and Lis-Bibliometrics Committee Chair Elizabeth Gadd provides the following summary of the Metrics SIG session at this year’s ARMA Annual Conference.

There are many stakeholders in the world of research evaluation by numbers, and the Metric Tide Report did a great job of teasing out who should be doing what. At the

 ARMA Conference

this year, the

 Metrics Special Interest Group

Who is responsible for responsible metrics?

Like most institutions, the release of the

 Metric Tide report

in July 2015 came at a busy time for us. We were in the midst of a REF open access campaign, continuing our work to embed our CRIS (Pure) into institutional processes, and thinking about how to increase our research data management provisions without breaking the bank.

The metric tidal wave – guest post by Karen Clews

The Bibliomagician blog

seeks to provide comment and guidance for practitioners engaging with bibliometrics.  We now are seeking an enthusiastic volunteer to take

 The Bibliomagician Blog

on to the next level!

The Bibliomagician seeks honorary Blog Manager

Welcome to the blog run by ordinary research support professionals within universities who are called upon to perform extraordinary magic with biblio (publication) metrics (numbers).

Think of a number, any number…

Greenwich Research and Enterprise Location:  Greenwich Salary:  £38,183 to £46,924 plus £4546 London weighting Contract Type:  Open Closing Date:  Monday 20 March 2017 Interview Date:  To be confirmed Reference:  1346 Greenwich Research and Enterprise (GRE) is the University’s central office responsible for developing a supportive research culture and establishing links with industry and enterprise. GRE works across four &#8230; <a href="https://thebibliomagician.wordpress.com/2017/02/21/job-opportunity-university-of-greenwich-is-looking-for-a-research-outputs-manager/" class="more-link">Continue reading "Job Opportunity: University of Greenwich is looking for a Research Outputs&#160;Manager!"</a>

Greenwich Research and Enterprise


 Location:

Greenwich

 Salary:

£38,183 to £46,924 plus £4546 London weighting

 Contract Type:

Open

 Closing Date:

Monday 20 March 2017

 Interview Date:

To be confirmed

 Reference:

1346 Greenwich Research and Enterprise (GRE) is the University’s central office responsible for developing a supportive research

Job Opportunity: University of Greenwich is looking for a Research Outputs Manager!

I love my job.  I work as a “Research Policy Manager (Publications)” at Loughborough University.  And I spend my time understanding and advising on how we can improve the quality and visibility of our research.  However the strategies for achieving this aren’t as straightforward as you might think.  And increasingly I feel like I’m leading a double life, both seeking to play and change the scholarly communication game.

My double life: playing and changing the scholarly communications game.  By Lizzie Gadd

In this post Prof. Henk Moed outlines why the term evaluative framework refers to a specification of the qualitative principles and considerations that provide guidance to a concrete assessment process. In her latest blog post: There’s no such thing as a bad metric, Lizzie Gadd calls on those who feel committed to the responsible use &#8230; <a href="https://thebibliomagician.wordpress.com/2018/02/06/in-responsible-use-of-metrics-an-evaluative-framework-is-essential-but-cannot-be-grounded-in-informetric-research/" class="more-link">Continue reading "Responsible use of metrics: an evaluative framework is essential, but cannot be grounded in informetric research &#8211; Guest post by Henk&#160;Moed"</a>

Responsible use of metrics: an evaluative framework is essential, but cannot be grounded in informetric research – Guest post by Henk Moed

Job Reference Number: UOS015654 Job Title: Library Scholarly Communications Manager Salary: Grade 8 £39,324-£46,924 per annum with potential to progress to £52,793 through sustained exceptional contribution

 Closing Date: 31st March 2017

Summary: Copyright advisory and advocacy services form a critical component of the infrastructure necessary in advancing teaching and learning in the digital age.

Job Opportunity: University of Sheffield is looking for a Library Scholarly Communications Manager!

In the light of the Wellcome Trust’s new responsible metrics mandate Lizzie Gadd reports on the findings from the Lis-Bibliometrics 2018 Responsible Metrics State-of-the-Art survey.

Who will welcome Wellcome?  Results of the 2018 responsible metrics state-of-the-art survey

Last month, Barack Obama published an article in the journal JAMA discussing progress to date with The Affordable Care Act – or Obamacare – and outlining recommendations for future policy makers. Obama’s article was picked up in the press and across social media immediately.

Why should a bibliometrician engage with altmetrics? Guest Post by Natalia Madjarevic

The Glucksman Library in the University of Limerick in Ireland has a vacancy for Head Research Services (Multiannual). As a new initiative the Head, Research Services will develop and deliver a range of research services at all stages of the research life cycle in the context of Open Science initiatives and ongoing developments in scholarly communication.

Job Opportunity: The University of Limerick is looking for a Head Research Services!

In September 2018, Lizzie Gadd and Ian Rowlands published the findings from a Lis-Bibliometrics survey of bibliometric practitioners as to how they’d like to see bibliometric and altmetric tools improve. The survey was aimed at starting a dialogue with the supplier community around how these tools might develop to better meet end-user needs.

Three things survey findings: supplier formal response

Katie Evans reflects on how their principles of research assessment and management are being applied at the University of Bath – one year on.

What difference does having a responsible metrics statement make? And how do you tell if a statement is doing what you need it to do? We published the University of Bath’s Principles of research assessment and management in March 2017.

What difference does a responsible metrics statement make?

Every year for the last three years I have ‘taken the temperature’ of the HE community regarding responsible metrics using a short survey.  The surveys have been advertised to the Lis-Bibliometrics Forum and the ARMA Metrics Special Interest Group. Back in 2015 I was keen to understand whether and how universities were responding to the San Francisco Declaration on Research Assessment (DORA).