A validated search filter is one which has had its performance tested against a known record set to verify that it reliably retrieves relevant results. This allows both you and your users to have confidence in the validity of the search and understand the limits of its performance. On this page you will find a list of the general steps to take and resources that you can use to develop a validated search filter.
General Steps for Creating a Validated Filter
Define the topic for the filter: determine what is and is not in scope of the filter
Identify a Gold Standard Set: the set of records that should be captured by the filter you are developing and that you can use to create and test the filter
Divide the Gold Standard Set into a Development and Validation Set: the Development Set is the set of records that you can use to develop the search filter; the Validation Set is the set of records that you can use to validate the filter
Develop the filter: identify the relevant search fields and terms using the Development Set
Validate the filter: run the filter and assess its performance using the Validation Set.
Example: test for Precision by calculating the percentage of out all records retrieved in the search results; test for Recall by calculating the percentage of records from the Validation Set in the search results
There are other metrics that you can use to test the filter (i.e. number needed to read, F1 score, etc.).
Visit the ISSG page on Methods of Developing Filters for more details.
Disseminate the filter: share the filter with other potential users
Resources
Ayiku L, Craven J, Hudson T, Levay P. How to Develop a Validated Geographic Search Filter: Five Key Steps. EBLIP [Internet]. 2020 Mar. 13;15(1):170-8. doi: https://doi.org/10.18438/eblip29633
This article provides a step-by-step approach to developing a validated filter, with a focus on geography
Bak, G., Mierzwinski-Urban, M., Fitzsimmons, H., Morrison, A. and Maden-Jenkins, M. (2009), A pragmatic critical appraisal instrument for search filters: introducing the CADTH CAI. Health Information & Libraries Journal, 26: 211-219. doi: https://doi.org/10.1111/j.1471-1842.2008.00830.x
This checklist is a method for filter validation methodology appraisal.
Cooper, C., Varley-Campbell, J., Booth, A. et al. Systematic review identifies six metrics and one method for assessing literature search effectiveness but no consensus on appropriate use. Journal of Clinical Epidemiology [Internet], 2020 Nov. 11. doi: https://doi.org/10.1016/j.jclinepi.2018.02.025
This article defines the metrics used to evaluate the effectiveness of a search.
Franco JVA, Garrote V, Vietto V, Escobar Liquitay CM, Solà I. Search strategies (filters) to identify systematic reviews in MEDLINE and Embase. Cochrane Database Syst Rev. 2020 Jul 3;2020(7):MR000054. doi: https://doi.org/10.1002/14651858.MR000054
This paper includes a detailed checklist for the appraisal of validated filters.
Frazier JJ, Stein CD, Tseytlin E, Bekhuis T. Building a gold standard to construct search filters: a case study with biomarkers for oral cancer. J Med Libr Assoc. 2015 Jan;103(1):22-30. doi: https://doi.org/10.3163/1536-5050.103.1.005
This study describes an alternative method to develop a gold standard set.
Hausner, E., Waffenschmidt, S., Kaiser, T. et al. Routine development of objectively derived search strategies. Syst Rev 1, 19 (2012). doi: https://doi.org/10.1186/2046-4053-1-19
Hausner et al describe an approach to the objective development of search strategies.
Lagisz M, Yang Y, Young S, Nakagawa S. A practical guide to evaluating sensitivity of literature search strings for systematic reviews using relative recall. Research Synthesis Methods. 2025;16(1):1-14. doi: https://doi.org/10.1017/rsm.2024.6
Lagisz et al provide a tutorial for sensitivity assessment of search strings.
Prady, S.L., Uphoff, E.P., Power, M. et al. Development and validation of a search filter to identify equity-focused studies: reducing the number needed to screen. BMC Med Res Methodol 18, 106 (2018). doi: https://doi.org/10.1186/s12874-018-0567-x
Prady et al. contrast specific and nonspecific search strategies and how each performs in validation.
Sampson, M., Zhang, L., Morrison, A. et al. An alternative to the hand searching gold standard: validating methodological search filters using relative recall. BMC Med Res Methodol 6, 33 (2006). doi: https://doi.org/10.1186/1471-2288-6-33
Sampson et al. discuss the development of a gold standard via the relative recall method, without the need for hand searching.
Text mining and thesauri [Internet]. LSSR Group; 2025 [updated 24 July 2025]. Available from https://sites.google.com/view/searchresourceslib/tools/text_mining
This LSSR page includes a number of resources that can be used for filter development.
Tompson, L. Testing filter term performance in PsycINFO to identify evidence syntheses in crime reduction, using the relative recall method. J Exp Criminol 15, 453–467 (2019). doi: https://doi.org/10.1007/s11292-019-09359-0
This paper outlines diverse approaches to subjective filter development.
Wafford QE, Miller CH, Wescott AB, Kubilius RK. Meeting a need: development and validation of PubMed search filters for immigrant populations. J Med Libr Assoc. 2024;112(1):22-32. doi: https://doi.org/10.5195/jmla.2024.1716
This study discusses the development of a validated filter for a complex search topic.
This page was last updated on 7 December 2025.