ISSG structured abstract for:
Boynton J, Glanville J, McDaid D, Lefebvre C. Identifying systematic reviews in MEDLINE: developing an objective approach to search strategy design. Journal of Information Science 1998;24(3):137-54.
Structured abstract prepared by: Sue Bayliss
Objective:
To use word frequency analysis to design a highly sensitive search strategy to retrieve systematic reviews and meta-analyses from MEDLINE.
Methods:
The authors identified a quasi-gold standard (GS) of 288 papers via a handsearch of 6 high impact factor journals and an electronic search of MEDLINE (Ovid). The search terms for the filters were generated using word frequency analysis – appearance of words in specific fields (ti, ab and subject indexing).
Results:
Eight filters of varying sensitivity and precision are presented as options depending on search requirements. Strategy E is suggested as best suited to the busy searcher offering precision of 79% with sensitivity of 29%, while the thorough searcher interested in sensitivity would be directed towards Strategy J offering a sensitivity of 98% and precision 20%.
Discussion:
The authors discuss a number of limitations – the gold standard is limited by using high impact journals and date restrictions (1992-1995) and possibly the inclusion/exclusion criteria used. Using more sophisticated frequency analysis software may produce strategies which are still highly sensitive but could also achieve higher levels of precision.
ISSG commentary:
The size of the gold standard is relatively small. Furthermore the same dataset was used to derive and validate the strategies, which could introduce bias. However, to test the validity of the objective approach to search-term generation the strategies were tested against three strategies not based on word frequency generation and against other published strategies (Hunt and McKibbon and NHS CRD).