2014 JQI

Summary

The JQI is constructed based on two main metrics: the Article Influence (AI) percentile and the national rankings of economics and business journals in the United Kingdom (ABS) and France (CNRS).  The index also utilizes information from the national list of Australian (ABDC) to create a comprehensive list of journals, as well as the journal rankings by peer institutions (see below for a description of the data sources).

Here is the 2014 JQI in the form of a Google sheet.  Users can filter and search for journals.

Interpretation

The JQI scores range from 0 to 10 and covers 2,869 peer-reviewed journals in business and economics.  Journals that are externally validated by the metrics receive a score greater than 0 while journals that are on the list (recognized as possible outlets for publication) but without external validation receive a 0.  Approximately 43% of the journals on the list receive a score of 0 (peer-reviewed journals without external validation).  

Journals that contribute substantially to scientific knowledge (high AI percentile) and are recognized highly (high ranks on ABS/CNRS) will have high scores on the JQI.  Journals that do not contribute as much to scientific knowledge or are not as widely regarded receive scores closer to 0.  Journals will have intermediate scores if they are in the middle of the AI and ABS/CNRS scales or in cases where there is a disagreement between the two metrics.

Data

The underlying metrics used are:

• Article Influence (AI) from the 2012 Journal Citation Reports (JCR).

  • Scores are converted to percentiles within the JCR.  

• The 2010 rankings in the Academic Journal Guide (AJG) created by The Chartered Association of Business Schools (ABS) (United Kingdom)

  • 5 categories of journals are converted to a 5 point scale (5 high)

• The 2011 rankings in the Categorization of Journals in Economics and Management by the Comite National de la Recherche Scientifique (CNRS) (France)

  • Categories are converted to a 5-point scale.

• Peer institution rankings

  • Rankings from Florida International University (use Financial Times list), Texas Tech, University of Connecticut, University of Memphis, University of Oklahoma, University of Pittsburgh, Washington State University.

The 2013 journal list of the Australian Business Deans Council (ABDC) is also utilized to help create the "base" list for the JQI (the exact rankings of the journals in the ABDC are not used in the JQI calculation).

Methodology (1.0) basics

We begin by merging the ABDC, ABS, CNRS, peer lists and the AI list.  From that, we keep all journals that appear on at least one of the ABDC, ABS, CNRS or peer lists.  In practice, this is the journals on the ABDC list which is by far the most comprehensive list.  This effectively provides the universe of economics and business journals (the remaining journals on the AI list being from outside these disciplines).  Currently, this provides a list of 2,869 journals.

We then convert both the ABS and CNRS into 5-point scales where 5 represent the extraordinarily recognized journals.  Due to missing data from journals not appearing on each list, we construct a measure Scale5 as the average of the two 5-point scales or the available scale when only one is available.   Scale5 is set to 0 if missing.  We convert the AI score to a percentile (AIP) of the entire distribution (science and social science).  We use the entire distribution because it will more easily allow future interdisciplinary work to be evaluated.  AIP is set to 0 if missing.

In the simple form, the Journal Quality Index is calculated as: JQI = Scale5 + (AIP)/20
which ranges from 0 to 10.  A score of 0 means that there is no outside validation of the journal quality.  Any journal with a score greater than 0 does have external validation, with varying ranges of quality.  

The primary problem with this simple calculation is that there are some journals for which there is only information on either  or  which will have 0 for part of the calculation.  To account for this, missing metrics are imputed using the available Article Influence percentile, ABS/CNRS rating, and the peer institution lists.  These imputations also account for differences in the metrics across disciplines.  Two imputations are calculated and the final score is the max of the non-imputed score or the average of the two imputations.  Thus, no journal is made worse by the imputation.  The details of the imputation are described here.  In practice, the imputation procedure has little effect on the JQI scores, with over 90% of journals without missing data having an imputed value less than 1 point different than the simple index.

Journals not on the list 

Journals that are not on the list will tend to be either: 1) very new in which case there are not yet external metrics used to construct the index and has not been recognized on any economics or business list, or 2) outside the typical universe of economics and business research.  In the latter case, which would incorporate a wide range of interdisciplinary work, journals can be generally placed on the JQI if they are one of the 8,000 or so science or social sciences with an article influence score.