Libraries
 

Contact Information
  
last updated: 3/5/2013

Research Metrics: Impact Factors, Citation Counts, h-index, and Altmetrics

Introduction

Following is a brief introduction to several popular research metrics. Note that these metrics are primarily used by those in the sciences, as humanities and arts scholars have different ways of evaluating the quality of research. Also keep in mind that each metric has its limitations, and all metrics can be "gamed" (artificially inflated). ("Beware the impact factor," Nature Materials, 2013; "What's Wrong with Citation Counts?" D-Lib Magazine, 2009; "A Case Study in Anti-Gaming Mechanisms for Altmetrics: PLoS ALMs and DataTrust," Altmetrics12 Workshop, 2012)


Impact Factor

For a given journal, the impact factor describes the average number of citations received per paper published in that journal during the two preceding years. The journal impact factor (JIF) is currently a de facto measure of publication quality for many disciplines' promotion and tenure process.


Over the past twenty years, the JIF has increasingly been called into question as a valid primary measure of publication quality for the following reasons:

  • The JIF is only available for some disciplines, primarily in the biomedical and physical sciences.
  • JIFs are slow to accumulate and update.
  • JIFs measure the quality of a journal, not of the individual articles contained therein. Theoretically, a low impact factor journal could publish better articles than a high impact factor journal.
  • JIFs do not apply to digital scholarship such as preprints shared on ArXiv, web-based projects like the Chymistry of Issac Newtwon, or data shared in ICPSR.
  • Journals from English-speaking countries are favored by ISI, the company who uses secret calculations to determine JIFs.

Resources for Finding Journal Impact Factors

Citation Counts and the h-index

For a journal article or other scholarly output, a citation count is the number of times said output has been cited in a publication. Citation counts are another popular measure of publication quality used in many disciplines' promotion and tenure processes.


While more granular than the journal impact factor and therefore more relevant to determining the impact of a specific article, citation counts are slow to accumulate. They also lack context.


A researcher's h-index is a single number that attempts to quantify the impact of a scholar's most cited works. Put simply, a scholar with an index of h has published h papers each of which has been cited in other papers at least h times. The h-index is considered a superior measure to citation counts as cannot be skewed by single, well-cited papers and also can be used to compare individuals, departments, programs, or any other group of scientists (Marnett 2010).


Resources for Finding Citation Counts and h-indexes

Altmetrics

Altmetrics are measures taken from the popular and scholarly social web that determine how a scholarly output has been shared. They are often combined with usage statistics (downloads and page views) to paint an illustration of scholarship's impact beyond the academy. Altmetrics can serve as a good supplemental set of metrics when paired with traditional measures such as journal impact factors, citation counts, and h-indexes.


Resources for Finding Altmetrics

  • ImpactStory: Use this web-based application to collect metrics for your diverse research outputs from across the web.
  • Altmetric.com: A subscription service available to individuals and research teams that tracks altmetrics and provides robust reporting functionality for your scholarship.

Other Ways to Assess Research Impact

  • Assessing the Impact of Research: While this framework was developed to help assess impact of academic biomedical research, much of it can be used by researchers of all disciplines.


last updated: 3/5/2013