Bramer 2017

Appraisal of: "Bramer, W.M., Rethlefsen, M.L., Kleijnen, J., & Franco, O.H. (2017). Optimal database combinations for literature searches in systematic reviews: a prospective exploratory study. Systematic Reviews, 6:245 ."

Reviewer(s): 

Deirdre Beecher

Alan Lovell

MS Copilot

Full Reference:
Bramer, W.M., Rethlefsen, M.L., Kleijnen, J., & Franco, O.H. (2017). Optimal database combinations for literature searches in systematic reviews: a prospective exploratory study. Systematic Reviews, 6:245. DOI: 10.1186/s13643-017-0644-y. 

Short description: 

This prospective exploratory study investigates which combinations of bibliographic databases yield the highest recall in systematic review searches. The authors analyzed 58 systematic reviews conducted at Erasmus MC, comparing the actual retrieval of included references across databases such as Embase, MEDLINE, Web of Science, and Google Scholar. They calculated recall, precision, and number needed to read (NNR) for individual and combined databases. 

The study found that 17% of included references were retrieved by only one database, with Embase contributing the most unique references. The optimal combination—Embase, MEDLINE, Web of Science, and Google Scholar—achieved 98.3% overall recall and 100% recall in 72% of reviews. The authors estimate that 60% of published systematic reviews fail to reach 95% recall due to insufficient database coverage. They recommend searching these four databases as a minimum for biomedical systematic reviews, with additional subject-specific databases used when appropriate. .

Limitations stated by the author(s): 

• The study did not assess whether missing references would have altered the conclusions of the reviews.  

• Findings may not generalize to all systematic reviews due to topical differences and searcher expertise.  

• Google Scholar’s limitations (e.g., lack of truncation, proximity operators, and search history) make it difficult to use effectively.  

• The study relied on searches conducted by a single experienced information specialist, which may not reflect typical practice. 

Limitations stated by the reviewer(s): 

• The study focuses on recall and database performance but does not evaluate the impact of missed references on review outcomes.  

• The reliance on a single searcher may introduce bias or limit reproducibility.  

• The study does not fully explore the trade-off between increased recall and the burden of screening more results (higher NNR). 

Study Type: 

Prospective Exploratory Study (Quantitative Evaluation of Search Strategy Performance)  

Related Chapters: 

Clinical effectiveness

Tags: 

• Search practices

• Databases