Hubbard 2022

Appraisal of: Hubbard W, Walsh N, Hudson T, Heath A, Dietz J, Rogers G. Development and validation of paired MEDLINE and Embase search filters for cost-utility studies. BMC Med Res Methodol. 2022;22(1):310. 


Reviewer(s): 

Julie Glanville

Dave Kaunelis


Full Reference: 

Hubbard W, Walsh N, Hudson T, Heath A, Dietz J, Rogers G. Development and validation of paired MEDLINE and Embase search filters for cost-utility studies. BMC Med Res Methodol. 2022;22(1):310. 

Short description: 

This paper describes the development of twin filters to identify records reporting cost-utility analyses (CUAs) within Embase and MEDLINE. The authors established a priori that they wished the filters to balance sensitivity and precision with a target sensitivity of at least 90%. The gold standard was created by obtaining eligible studies from NICE guidelines. The filters were developed using word frequency analysis and tested on a development set of 115 records. Candidate filters were validated on a validation set of 370 records. Filter performance was compared against the authors' pre-existing broad economic search strategies in a randomly selected single year (2010).

The paired sensitive filters achieved 100% sensitivity but only involved screening 20% of the records that the pre-existing broad strategy retrieved. The precise filter achieved 97.6% sensitivity with an even better improvement in precision. The free text filters are suggested as potential base filters to use within databases such as PsycINFO.

This is the first filter to be focused on identifying cost-utility analyses specifically.

Limitations stated by the author(s): 

The only CUAs represented were those measuring utilities in terms of QALYs, but the authors tested the retrieval of non-QALY studies and found the sensitive and precise filters found studies using DALYs, although adding DALY terms in would be advisable when seeking such studies.

The filters missed NIHR technology assessment monographs, so separate searches of that journal are recommended.

Additional studies are likely to be retrieved in databases other than MEDLINE and Embase.

The authors did not include terms in the author keywords field or the journal name in the filter development. 

The authors acknowledged the limitations of the relative recall method that they used to develop the filter but suggested that the extensive searching conducted to produce NICE guidelines would minimise some of the impacts of the approach.


Limitations stated by the reviewer(s): 

The performance is tested against records retrieved by another filter in just one year (2010) which is relatively old.

People may not search both databases as pairs in real life and data are not provided in the paper for the performance of the filters individually.


Study Type: 

Single Study

Related Chapters: 

Costs and economic evaluation.

Tags: