Sampson 2006
Appraisal of: "Sampson M, Zhang L, Morrison A, Barrowman NJ, Clifford TJ, Platt RW, et al. An alternative to the hand searching gold standard: validating methodological search filters using relative recall. BMC Med Res Methodol. 2006;6(33)"
Reviewer(s):
Kath Wright
Sonia Garcia
Full Reference:
Sampson M, Zhang L, Morrison A, Barrowman NJ, Clifford TJ, Platt RW, et al. An alternative to the hand searching gold standard: validating methodological search filters using relative recall. BMC Med Res Methodol. 2006;6(33)
Short description:
This paper explores the relative recall technique to develop a gold standard for use in evaluating search filters. Filters or ‘hedges’ are search strategies used in conjunction with a topic search to focus on those records which are the focus of the filter. Validating a filter requires the existence of a ‘gold standard’ of records known to be of the study type required. However, developing a gold standard by handsearching can be time consuming. The authors identified 105 Cochrane Reviews that had used the Cochrane Highly Sensitive Search Strategy to identify studies. The included studies in those reviews were gathered to form a gold standard. Three variants of the Highly Sensitive Search Strategy (HSSS) filter were tested in terms of their ability to retrieve gold standard records in MEDLINE. Sensitivity ranging from 0.98 to 0.91 was achieved.
Limitations stated by the author(s):
The feasibility of this method will depend on the number of available systematic reviews using the methodology of interest. The authors acknowledge criticisms of sensitivity (recall) and precisions as measures of search performance.
Limitations stated by the reviewer(s):
No additional limitations detected by the reviewers.
Study Type:
Single study
Related Chapters:
Tags: