Embedded Files
Appraisal of: "Cooper, C., Varley-Campbell, J., Booth, A., Britten, N., & Garside, R. (2018). Systematic review identifies six metrics and one method for assessing literature search effectiveness but no consensus on appropriate use. Journal of Clinical Epidemiology, 99, 53–63 ."
Appraisal of: "Cooper, C., Varley-Campbell, J., Booth, A., Britten, N., & Garside, R. (2018). Systematic review identifies six metrics and one method for assessing literature search effectiveness but no consensus on appropriate use. Journal of Clinical Epidemiology, 99, 53–63 ."
Reviewer(s):
Deirdre Beecher
Alan Lovell
MS Copilot
Full Reference:
Cooper, C., Varley-Campbell, J., Booth, A., Britten, N., & Garside, R. (2018). Systematic review identifies six metrics and one method for assessing literature search effectiveness but no consensus on appropriate use. Journal of Clinical Epidemiology, 99, 53–63. https://doi.org/10.1016/j.jclinepi.2018.02.025.
Short description:
This systematic review aimed to identify and evaluate the metrics and methods used to assess the effectiveness of literature searching in health and allied fields. The authors screened 9,126 records and included 50 studies that compared supplementary search methods (e.g., handsearching, citation chasing) with bibliographic database searching. Six metrics—sensitivity, specificity, precision, accuracy, number needed to read (NNR), and yield—and one method—capture-recapture—were identified.
The review found that sensitivity was the most commonly used metric (90% of studies), followed by precision (80%) and specificity (68%). However, only 34% of studies defined thresholds for effectiveness, and just over half reported confidence intervals. The authors concluded that there is no consensus on how to define or measure search effectiveness and recommended clearer reporting, validation of search strategies, and use of thresholds and confidence intervals in future evaluations.
Limitations stated by the author(s):
No formal quality appraisal of included studies due to lack of suitable tools.
Limited to studies in health and allied topics using MEDLINE and EMBASE.
Many included studies lacked clear definitions of effectiveness or thresholds.
Poor reporting and validation of search strategies in included studies.
Limitations stated by the reviewer(s):
The review does not assess the impact of missed studies on review outcomes.
The study does not explore the practical implications of effectiveness metrics for decision-making in real-world reviews.
The absence of consensus on metric use limits the applicability of findings.
Study Type:
Systematic Review
Related Chapters:
Tags:
Literature searching
Page updated
Report abuse