Summaries of Text Mining Research

Hausner E et al J Clin Epidemiol 2016

• Compared traditional searching approach to approaches using an objective strategy development approach using text mining tools such as WordStat

• The algorithm used for the objective approach is quite detailed

• Developed searches for MEDLINE for 5 reviews

• Objective approach was more sensitive than conceptual approach: 97% vs. 75%

• Not a great difference in the time taken: The objective approach took mean 16 hours and traditional approach took mean 17 hours.

• Quite a small sample size and not all reviews were complex

Paynter et al. J Clin Epi 2021

• 7 review searches – 1 librarian created MEDLINE search strategies using TM and other did usual practice

• Usual practice was more sensitive (92% vs 84.9% but confidence intervals overlapped) and had lower NNR (83 vs 90) than TM

• TM tool less time (5 hours vs 12 hours) and saved more time for complex questions

• For the three complex topics TM searches were faster and less sensitive than usual practice searches but identified unique eligible citations

• TM is not yet ready to be used as sole process for developing SR searches but time savings may be useful particularly in context of scoping or rapid reviews

Scells H et al. Information Retrieval Journal 2021

• Experiments conducted on the CLEF TAR 2018 set of 75 diagnostic test accuracy systematic reviews

• Compared their automatic query formulation approach to the original review searches

• None of the automated methods can outperform the original review searches

• MeSH was important for maximising recall

• More effective queries contain both phrases and MeSH terms, but CHOICE of phrase is important

• Combination of terms into concepts is very important

• The more seed studies used the better the recall

• Manual refinement of conceptual approach automatic strategies improved recall

O’Keefe et al. Research Synthesis methods 2022

• Case study of using 16 text mining tools to design a search strategy for a DTA review

• Used 5 seed papers to explore the resources and rate their usability and usefulness

• Using the tools and the terms they generated identified 11 additional articles relevant to the review

• “Extremely useful tools”: Text Analyzer, Yale MeSH Analyser

• “Useful tools”: Anne O’Tate, Voyant, PubReMiner

• However, the focus seems to be on ease of use which means that sophisticated tools which have a learning curve may be down-rated

References

Hausner E, Guddat C, Hermanns T, Lampert U, Waffenschmidt S. Prospective comparison of search strategies for systematic reviews: an objective approach yielded higher sensitivity than a conceptual one. J Clin Epidemiol. 2016.

O'Keefe H, Rankin J, Wallace SA, Beyer F. Investigation of text-mining methodologies to aid the construction of search strategies in systematic reviews of diagnostic test accuracy-a case study. Res Synth Methods. 2023;14(1):79-98.

Paynter RA, Featherstone R, Stoeger E, Fiordalisi C, Voisin C, Adam GP. A prospective comparison of evidence synthesis search strategies developed with and without text-mining tools. J Clin Epidemiol. 2021:S0895-4356(21)00085-8.

Scells H, Zuccon G, Koopman B. A comparison of automatic Boolean query formulation for systematic reviews. Information Retrieval Journal. 2021;24(1):3-28.