Overview of reviews

Some topic areas are notable for already having an abundance of published systematic reviews and meta-analyses but there still may be uncertainty about the extent to which these reviews can be used to guide clinical recommendations. Systematic reviews often need updating and not all reviews are of equal quality and may be affected by various biases. For example, some reviews differ in terms of how inclusive they are of different types of study reports (potentially leading to publication bias), or search strategies might not have been executed in a transparent or replicable fashion, potentially leading to reviewer selection bias. In these circumstances it can be advantageous to undertake what is called a ‘review of reviews’. The task here is to assemble and summarise the existing evidence on a particular topic area, using systematic reviews as the primary study design. Systematic review methods can be applied here as the same principles apply to reviewing existing reviews as they do to reviewing primary study designs such as RCTs. For example, you would still be expected to use an approach underpinned by the PICO(S) formula. However, because the unit of study is now a review, methods must be adapted accordingly, particularly in terms of the search strategy (it is not usual to try to identify new primary studies), study selection criteria, data collection, methodological assessment and data synthesis. It is normally expected that dissertations based on overviews of reviews will include the following:

• a focused and addressable research question, expanded into aims and objectives as appropriate;

• background section explaining the topic and justification of the need for the overview;

• a methods section incorporating justification of overview methods and citing of appropriate methodological literature, with the following sub-sections;

  • • pre-specified selection criteria for primary studies (reviews/meta-analyses);

    • a comprehensive search strategy;

    • standardised data collection strategy for each included review;

    • standardised methodological assessment plan for each included review using an appropriate tool that either qualitatively and/or quantitatively assesses quality and risk of bias (for example AMSTAR-2 or ROBIS tool);

    • plans for a structured data synthesis (which may be purely narrative or could include meta-analysis to compare, indirectly, treatment effects of interventions of interest across reviews);

• results section which should include the following:

  • • results of search strategy;

    • description of included reviews including summary of methodological quality and risk of bias;

    • estimates of effect for all pre-specified outcomes.

• discussion section which should include a summary of main findings, an account of resolution/remaining uncertainty in relation to the review question, a summary of general and specific limitations of included reviews, strengths and limitations of the overview, context of findings in relation to existing literature, recommendations for future research and clinical or other professional practice, presentation of a dissemination plan (if appropriate) and concluding comments;

• there must be evidence of critical and analytical treatment of primary material throughout the dissertation;

• overview methods, processes and findings must be reported in a transparent way.

Example research questions:

• What is the evidence from Cochrane systematic reviews on the efficacy and safety of non-pharmacological and pharmacology interventions to manage post-traumatic stress disorder?

• What is the evidence from Cochrane systematic reviews on the efficacy and safety of non-pharmacological and pharmacology interventions to manage pain in labour?

• What is the evidence for evaluation of the impact of financial incentives on healthcare professional behaviour and patient outcomes?

• What is the evidence from Cochrane reviews of the efficacy and safety of pharmacological interventions designed to assist smoking cessation attempts?

The above examples are available from the Cochrane Library. Students may view other overviews of reviews by doing a simple search on the Cochrane Library using the search term ‘overview’.

Useful references:

• Shea B J, Reeves B C, Wells G, Thuku M, Hamel C, Moran J et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both BMJ 2017; 358 :j4008 doi:10.1136/bmj.j4008

• Whiting P, Savović J, Higgins JP, Caldwell DM, Reeves BC, Shea B, Davies P, Kleijnen J, Churchill R; ROBIS group. ROBIS: A new tool to assess risk of bias in systematic reviews was developed. J Clin Epidemiol. 2016 Jan;69:225-34. doi: 10.1016/j.jclinepi.2015.06.005. Epub 2015 Jun 16. PMID: 26092286; PMCID: PMC4687950.

• Kung, J., Chiappelli, F., Cajulis, O.O., Avezova, R., Kossan, G., Chew, L., Maida, C.A. (2010). From systematic reviews to clinical recommendations for evidence-based health care: Validation of revised assessment of multiple systematic reviews (R-AMSTAR) for grading of clinical relevance. The Open Dentistry Journal, 4, 84-91.

• Peters LWH, Kok G, Ten Dam GTM, Buijs GJ, Paulussen TGWM. Effective elements of school health promotion across behavioral domains: a systematic review of reviews. BMC Public Health BMC Public Health 2009, 9:182 doi:10.1186/1471-2458-9-182. Available from: http://www.biomedcentral.com/1471-2458/9/182 (accessed April 2012).

• Page MJ, McKenzie JE, Kirkham J, Dwan K, Kramer S, Green S, Forbes A. Bias due to selective inclusion and reporting of outcomes and analyses in systematic reviews of randomised trials of healthcare interventions. Cochrane Database Syst Rev. 2014 Oct 1;10:MR000035. doi: 10.1002/14651858.MR000035.pub2.

• Cochrane Handbook for Systematic Reviews of Interventions. Available from http://training.cochrane.org/handbook (accessed Feb 2018). See Chapter 22, Overviews of reviews.