Skip to main content

Bibliometrics: Responsible Metrics & DORA

Learn how to track citations to your research and the limitations of using bibliometric indicators

Using Research Metrics Responsibly

Whilst bibliometric indicators can be a useful source of information when used responsibly, they are very narrow and limited measures and should not be used in isolation. Combining a wide range or "basket" of contextualised metrics across a range of appropriate dimensions can help to supplement or inform expert, qualitative peer review and assessment. A broad range of outputs and activities should be recognised as part of any research evaluation process, not just publications. This may include aspects such as supervision and mentorship, openness, peer review and editorial roles, contribution to economy, society and policy, awards & honours, public engagement activities etc.

Journal metrics should never be used to evaluate the quality or impact of an individual article or an individual researcher. 

San Francisco Declaration on Research Assessment

DORA

Initiated in 2012, the American Society for Cell Biology and a group of editors and publishers of scholarly journals drafted and circulated a declaration that recognises the need to improve the way in which the outputs of scientific research are evaluated.

"The Journal Impact Factor, as calculated by Thomson Reuters, was originally created as a tool to help librarians identify journals to purchase, not as a measure of the scientific quality of research in an article. With that in mind, it is critical to understand that the Journal Impact Factor has a number of well-documented deficiencies as a tool for research assessment. These limitations include:

  • citation distributions within journals are highly skewed
  • the properties of the Journal Impact Factor are field-specific: it is a composite of multiple, highly diverse article types, including primary research papers and reviews
  • Journal Impact Factors can be manipulated (or "gamed") by editorial policy
  • data used to calculate the Journal Impact Factors are neither transparent nor openly available to the public."

The Leiden Manifesto for Research Metrics

"To support researchers and managers, five experts led by Diana Hicks, professor in the School of Public Policy at Georgia Institute of Technology, and Paul Wouters, director of CWTS at Leiden University, have proposed 10 principles for the measurement of research performance: the Leiden Manifesto for Research Metrics published as a comment in Nature." - April 23rd 2015.

Research Culture at the University of Glasgow

Funding Agencies & Responsible Metrics

Many funding agencies are now DORA signatories, including Science Foundation Ireland and the HRB.

From 2021 Wellcome Trust will introduce a new open access policy that requires Wellcome-funded organisations to publicly commit to the principle of evaluating research based on the intrinsic value of the work rather than the venue it is published in: "For example, they can sign the San Francisco Declaration on Research Assessment, Leiden Manifesto or equivalent. We may ask organisations to show that they’re complying with this as part of our organisation audits."

Other Useful Resources on Responsible Research Metrics

Examples of Research Evaluation Statements / Positions