Measuring the impact of your research

There are many metrics that can be used to measure the impact of one’s research. While each metric looks at a different scope of research impact, there isn’t a single metric that illustrates all the aspects of research.

In June, Kristi Holmes, PhD, Associate Professor of Preventive Medicine and Director of Galter Health Sciences Library, participated as faculty at the training course on Health Research Impact Assessment at Alberta Innovates Health Solutions. During this course, Holmes provided guidance on measuring research impact through publications.

“It is critical for investigators to be able to understand the impact of their work and communicate that impact to the broader community,” Holmes said, “Publication data can be a good place to start this process, given the importance of communicating our work through the published literature.”

Choosing metrics

Choosing an appropriate metric is important for accurately portraying the impact of one’s research. Holmes recommends asking the following questions to help determine the appropriate metrics:

  • What level assessment will you look at?
  • Publication, individual scholar, a research group, institution, country, etc.
  • What do you want to examine?
  • Productivity, impact, collaboration, subject area, etc.
  • What is your overall purpose?
  • What data do you have available?
  • Who is your audience?

Common Indicators

Generally speaking, there are a few common indicators that can be used when leveraging publication data. For example, the number of publications can be used as a measure of knowledge production, citations can serve to indicate impact and broader visibility of the work, and co-authorship patterns can be used to demonstrate the level and types of collaboration.[1] The particular journal where the research is published can even be used as an indirect indicator of quality.

Three commonly discussed metrics include the h index, m index and Journal Citation Reports impact Factor (IF) Score. Each one is appropriate in different situations.

Metric

Calculation

Analysis of metric  

h index

#papers (h) that have received at least h citations

-Good for more established authors

-Combines publications and citation counts into a single metric

-Difficult to compare between disciplines

-Doesn’t factor in context or source of citations

-Can be applied on the individual or group level

m index

h index/# of years since first paper

-Variant of h index

-Useful for early-career authors, as it accounts for the number of years since the first paper

Journal Impact Factor score

Average number of times articles from the journal published in the past two years have been cited in the Journal Citation Reports year

-Does not provide metrics about a specific author or paper

-Does not predict scientific research impact

-This score should be used as a comparison tool between journals, not to compare authors or to measure an individual’s research impact

 

iCite: a new tool from NIH

A new tool for evaluating bibliometrics was recently created by the NIH, called iCite. iCite is easy to use: users upload the PubMed IDs of articles of interest and the tool then displays metrics on these papers such as the number of articles, articles per year, citations per year, and Relative Citation Ratio[2] (a field-normalized metric that shows the citation impact of one or more articles relative to the average NIH-funded paper).

Why does measuring research impact through publication data matter? 

Dissemination of research discoveries through the published literature is an important part of the scholarly process and data about publications can be a very powerful indicator about research. Publication data can help investigators build their research careers in many ways. It can be used to illustrate qualifications to undertake a new research project, justify a grant renewal, or show accomplishments for tenure or a promotion. In addition, publication data can also be used to highlight other aspects of the research or the study team, including domain impact, willingness to share research findings through Open Access publishing practices, as well as collaboration with authors in different areas of research or with those at other institutions.

“Publication data is a powerful tool to understand research impact, but it is just the tip of the iceberg” Holmes shares. “It is essential to go beyond the publication to understand and look for opportunities to communicate meaningful health outcomes to the public, policy makers, and other key stakeholders.”

For more information, contact Holmes or Galter Library’s Metric and Impact Core.


Posted: July 11, 2016

Back to the NUCATS Newsroom