Golder 2006
Appraisal of: "Golder S, McIntosh HM, Loke Y. Identifying systematic reviews of the adverse effects of health care interventions. BMC Med Res Methodol 2006;6:22."
Reviewer(s):
Carol Lefebvre
Sari Susanna Ormstad
Full Reference:
Golder S, McIntosh HM, Loke Y. Identifying systematic reviews of the adverse effects of health care interventions. BMC Med Res Methodol 2006;6:22.
Short description:
The aim of the study was to assess how easily one could identify systematic reviews of adverse effects in the two major databases of systematic reviews: the Database of Abstracts of Reviews of Effects (DARE) and the Cochrane Database of Systematic Reviews (CDSR). A gold standard set of records (270 systematic reviews) was identified from DARE and CDSR by carrying out searches for systematic reviews of adverse effects published from 1994 to 2005. The search strategies used a combination of text words in the title and abstract, subject headings (MeSH) and subheadings. In addition, DARE records in progress were hand searched. The search terms used to identify the systematic reviews were assessed for their usefulness in retrieving relevant records by measuring their sensitivity and precision. The results showed that ‘floating’ subheadings provided the highest sensitivity in both DARE and CDSR (85% and 64% respectively). All single search terms in CDSR resulted in very low precision (0% - 3%). Single MeSH terms provided the highest level of precision in DARE (67%-73%). The most sensitive search strategy in DARE used a combination of text words in the title and abstract, a MeSH term and ‘floating’ subheadings (94%). The precision of this strategy was 16%. In CDSR the most sensitive strategy used the ‘floating’ subheading ‘adverse effects’ and text word search ‘adverse near/20 objectives’ in the abstract field (79%). The precision of this strategy was 3%.
Limitations stated by the author(s):
Although DARE and CDSR are excellent sources of systematic reviews of adverse effects, not all reviews reported as being systematic are contained in these databases. Also, systematic reviews included in these databases tend to be of higher methodological quality, which may reflect better reporting and therefore better indexing. The low number of systematic reviews of adverse effects in CDSR (14) made the analysis of data and assessment of usefulness of individual search terms difficult. The search terms tested in the study were predefined from previous research and were not obtained by objective methods. However, papers not retrieved with the predefined search terms didn’t reveal many additional terms.
Limitations stated by the reviewer(s):
Even though this study provides the most up-to-date evidence on the topic of identifying systematic reviews of adverse effects in CDSR and DARE, readers should bear in mind that the study was published in 2006. Adverse effects terms may be increasingly prevalent in the title, abstract and indexing terms of records of studies published more recently compared to those published up until 2006. Readers may, therefore, wish to consider the appraisals of more recent work by Golder et al which address identifying single studies reporting adverse effects. Changes made in the two databases in recent years (such as new user interfaces and search functionality) may also influence the validity of the study findings. In addition to the limitation outlined above regarding the very low number of systematic reviews of adverse effects in CDSR (14), it should also be noted that reviews of adverse effects make up a relatively low percentage of the total number of reviews in DARE and an even lower percentage of the total number of reviews in CDSR and that this might have some bearing on the findings of the study. Also, it should be noted that the same set of search terms was used both to identify the systematic reviews for the gold standard set of records, and to measure the performance of single search terms and their combinations. This might have influenced the findings of this study.
Study Type:
Single study
This publication is related to Su Golder’s PhD Thesis “Evaluating and Optimising the Retrieval of Research Evidence for Systematic Reviews of Adverse Drug Effects and Adverse Drug Reactions” from 2013. The thesis is available from http://etheses.whiterose.ac.uk/4749/
Related Chapters:
Tags:
Safety
Controlled vocabulary
Text words