Summary
The JQI is constructed based on two main metrics: the Article Influence (AI) percentile and the national rankings of economics and business journals in the United Kingdom (ABS) and France (CNRS). The index also utilizes information from the national list of Australian (ABDC) to create a comprehensive list of journals (see below for a description of the data sources).
Note: The methodology for the 2020 JQI (version 1.1) is slightly changed from the 2017 version (version 1.0). The 2020 JQI no longer includes peer institutions which should make the list more universally useful and less specific to my institution. In practice, testing using the 2020 data shows that inclusion or exclusion of these peer institution metrics has little, if any, effect on the rankings of journals.
Here is the 2020 JQI list in the form of a Google sheet. Users can filter and search for journals.
Interpretation
The JQI scores range from 0 to 10 and covers approximately 3,100 peer-reviewed journals in business and economics. Journals that are externally validated by the metrics receive a score greater than 0 while journals that are on the list (recognized as possible outlets for publication) but without external validation receive a 0. Approximately 43% of the journals on the list receive a score of 0 (peer-reviewed journals without external validation).
Journals that contribute substantially to scientific knowledge (high AI percentile) and are recognized highly (high ranks on ABS/CNRS) will have high scores on the JQI. Journals that do not contribute as much to scientific knowledge or are not as widely regarded receive scores closer to 0. Journals will have intermediate scores if they are in the middle of the AI and ABS/CNRS scales or in cases where there is a disagreement between the two metrics.
Data
The underlying metrics used are:
Article Influence from the 2019 Journal Citation Reports (JCR).
Scores are converted to percentiles within the JCR.
The 2018 rankings in the Academic Journal Guide (AJG) created by The Chartered Association of Business Schools (ABS) (United Kingdom)
5 categories of journals are converted to a 5 point scale (5 high)
The 2020 rankings in the Categorization of Journals in Economics and Management by the Comite National de la Recherche Scientifique (CNRS) (France)
Categories are converted to a 5-point scale.
The 2019 journal list of the Australian Business Deans Council (ABDC) is also utilized to help create the "base" list for the JQI (the exact rankings of the journals in the ABDC are not used in the JQI calculation).
Methodology (1.1) basics
We begin by merging the ABDC, ABS, CNRS and the AI list. From that, we keep all journals that appear on at least one of the ABDC, ABS, or CNRS. In practice, this is the journals on the ABDC list which is by far the most comprehensive list. This effectively provides the universe of economics and business journals (the remaining journals on the AI list being from outside these disciplines). Currently, this provides a list of approximately 3,100 journals.
We then convert both the ABS and CNRS into 5-point scales where 5 represent the extraordinarily recognized journals. Due to missing data from journals not appearing on each list, we construct a measure Scale5 as the average of the two 5-point scales or the available scale when only one is available. Scale5 is set to 0 if missing. We convert the AI score to a percentile (AIP) of the entire distribution (science and social science). We use the entire distribution because it will more easily allow future interdisciplinary work to be evaluated. AIP is set to 0 if missing.
In the simple form, the Journal Quality Index is calculated as: JQI = Scale5 + (AIP)/20
which ranges from 0 to 10. A score of 0 means that there is no outside validation of the journal quality. Any journal with a score greater than 0 does have external validation, with varying ranges of quality.
The primary problem with this simple calculation is that there are some journals for which there is only information on either or which will have 0 for part of the calculation. To account for this, missing metrics are imputed using the available Article Influence percentile, and ABS/CNRS rating. These imputations also account for differences in the metrics across disciplines. Two imputations are calculated and the final score is the max of the non-imputed score or the average of the two imputations. Thus, no journal is made worse by the imputation. The details of the imputation are described here. In practice, the imputation procedure has little effect on the JQI scores, with over 90% of journals without missing data having an imputed value less than 1 point different than the simple index.
Journals not on the list
Journals that are not on the list will tend to be either: 1) very new in which case there are not yet external metrics used to construct the index and has not been recognized on any economics or business list, or 2) outside the typical universe of economics and business research. In the latter case, which would incorporate a wide range of interdisciplinary work, journals can be generally placed on the JQI if they are one of the 12,000 or so science or social sciences with an article influence score.