Other limits: language, date

Last updated: 12 March 2024

What's new in this update

No new research relevant to the chapter was identified during the March 2024 update. 

Introduction

The Cochrane Information Retrieval Methods Group have published an evidence-based chapter on search methods for the Cochrane Handbook (1) and accompanying technical supplement (4), which provides the basis for this summary alongside guidance produced by the Centre for Reviews and Dissemination (2), the Agency for Healthcare Research and Quality (AHRQ) (3) and EUnetHTA (6).

Limiting a search strategy by date

A date restriction should only be applied to the search strategy if there is a date restriction in the eligibility criteria (1). Limiting a search strategy by date may reduce the number of records retrieved for screening, but the eligibility criteria and search should only be restricted by date if there is a robust rationale for doing so. For example, if a healthcare intervention was only available after a certain date, limiting the search strategy to only retrieve studies reported from this date would be appropriate (1).  The PRISMA 2020 guideline (8) for reporting systematic reviews, and the PRISMA-S extension (9) for reporting literature searches in systematic reviews, require the justification for the use of any limits (including date) to be reported with reference to the eligibility criteria.

Applying date limits to the search strategy can also be an option if an existing search is being updated (1). When conducting an update search, searchers should be cautious about how date limits are applied. Limiting searches by publication date risks missing relevant records (2). If attempting to limit by date, an appropriate field (or fields) relating to date of entry into the database rather than publication date should be used (1,2,3,6). EUnetHTA also suggest two alternative options to applying date limits to the strategy. The first is to exclude all records retrieved from a database in the initial search (and identified via their accession number) via the use of NOT. The second alternative option is to download all the records from the update search (not restricted by date) and then to deduplicate these references against the records retrieved by the initial search (using EndNote for example) (6).

For databases where there is no update field, running the search without date limits is advised, using reference managing software to de-duplicate the returned records against the original search results. (2)

Limiting a search strategy by language

If the search strategy is restricted by language (for example, to English language studies only), there is a possibility of language bias (1,2,3,6). For topic areas where research from, or relating to, non-English speaking regions is of increased significance, the issue of limiting by language may be a particular concern. The PRISMA 2020 guideline (8) for reporting systematic reviews, and the PRISMA-S extension (9) for reporting literature searches in systematic reviews, require the justification for the use of any limits (including language) to be reported with reference to the eligibility criteria.   

Shenderovich et al studied methodological issues in systematic reviews which aimed to include evidence from low- and middle-income countries, using the example of a review of risk factors for child conduct problems and youth violence (5). The authors reported that 15% of the eligible studies were in a language other than English and therefore would not have been retrieved if English language search limits were applied. The impact of omitting the non-English studies on the conclusions of the review was not investigated.

Dobrescu et al conducted a systematic review with the aim of evaluating the impact of restricting literature searches and/or study inclusion criteria to English-language publications on the effect estimates, certainty of evidence ratings, and conclusions of systematic reviews (7).  The authors found that restricting in this way had little impact on the effect estimates and conclusions for most medical topics.  However, they found that complementary and alternative medicine reviews focusing solely on English-language studies could lead to different results and conclusions than language-inclusive reviews.  They could not identify any study assessing the impact of English-language restrictions on diagnostic test accuracy reviews.  The authors stated that implications were: that restricting search strategies and/or inclusion criteria to English-language publications seemed to be a viable approach for rapid reviews and systematic reviews with limited resources; evidence syntheses addressing complementary and alternative medicine topics should also include publications in languages other than English.

Current guidance recommends that search strategies should not be restricted by language (1,2,3). This is advised even if translation is not feasible (1,2,3). If translation is not possible, reviewers could file potentially relevant non-English studies in a ‘studies awaiting classification’ section, to inform readers of the review of the availability of other possibly relevant reports (1). An alternative option is for reviewers to exclude non-English language studies from the review, but make a list of potentially relevant studies which were excluded on the basis of language (2). This can help inform an assessment of the potential risk for language bias (2,3).

If a decision is made to apply language limits, the limits should only be considered in the individual databases where they can be reliably applied (6).

Reference list

(1) Lefebvre C, Glanville J, Briscoe S, Featherstone R, Littlewood A, Marshall C, Metzendorf M-I, Noel-Storr A, Paynter R, Rader T, Thomas J, Wieland LS. Chapter 4: Searching for and selecting studies. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.3 (updated February 2022). Cochrane, 2022. Available from www.training.cochrane.org/handbook.  [Publication appraisal]

(2) Systematic Reviews. CRD’s guidance for undertaking reviews in health care. Centre for Reviews and Dissemination, University of York; 2009.

(3) Methods Guide for Effectiveness and Comparative Effectiveness Reviews. AHRQ Publication No. 10(14)-EHC063-EF. Rockville, MD: Agency for Healthcare Research and Quality. 2014 update. [Publication appraisal]

(4)  Lefebvre C, Glanville J, Briscoe S, Featherstone R, Littlewood A, Marshall C, Metzendorf M-I, Noel-Storr A, Paynter R, Rader T, Thomas J, Wieland LS. Technical Supplement to Chapter 4: Searching for and selecting studies. In: Higgins JPT, Thomas J, Chandler J, Cumpston MS, Li T, Page MJ, Welch VA (eds). Cochrane Handbook for Systematic Reviews of Interventions Version 6.3 (updated February 2022). Cochrane, 2022.  Available from: www.training.cochrane.org/handbook.  [Publication appraisal] 

(5) Shenderovich Y, Eisner M, Mikton C, Gardner F, Liu J, Murray J. Methods for conducting systematic reviews of risk factors in low- and middle-income countries. BMC Med Res Methodol. 2016 Mar 15;16:32. [Publication appraisal]

(6) EUnetHTA JA3WP6B2-2 Authoring Team. Process of information retrieval for systematic reviews and health technology assessments on clinical effectiveness. Methodological Guidelines. Diemen (The Netherlands): EUnetHTA; 2019. [Publication appraisal]

(7) Dobrescu AI, Nussbaumer-Streit B, Klerings I, Wagner G, Persad E, Sommer I, Herkner H, Gartlehner G. Restricting evidence syntheses of interventions to English-language publications is a viable methodological shortcut for most medical topics: a systematic review. J Clin Epidemiol. 2021 Sep;137:209-217. doi: 10.1016/j.jclinepi.2021.04.012. Epub 2021 Apr 30. [Publication appraisal]

(8) Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, Shamseer L, Tetzlaff JM, Akl EA, Brennan SE, Chou R, Glanville J, Grimshaw JM, Hróbjartsson A, Lalu MM, Li T, Loder EW, Mayo-Wilson E, McDonald S, McGuinness LA, Stewart LA, Thomas J, Tricco AC, Welch VA, Whiting P, Moher D. The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. J Clin Epidemiol. 2021 Jun;134:178-189. doi: 10.1016/j.jclinepi.2021.03.001. Epub 2021 Mar 29. [Publication appraisal]