Costs and economic evaluation

Last updated: 6 March 2025

What's new in this update

We have included a case report of different search methods for finding utility values (Lister 2024) and an audit of approaches to identifying economic studies reported in NICE Single technology assessments (Misso 2024). A second study of including environmental impact in economic evaluations has been added (Silber 2024). An additional analysis of the utility of the ISPOR Presentations Database has been added (Lee 2024).

Introduction

This domain focuses on the importance of obtaining information about costs and outcomes as well as efficacy and effectiveness when evaluating new technologies. Economic evaluation is an important part of health technology assessment because it assists with priority-setting between different health technologies. An economic evaluation identifies, measures, values and compares the costs and outcomes of a technology with its relevant comparator.

This domain overlaps with the effectiveness domain and the organizational domain in the original EUnetHTA HTA core model (Lampe 2009, Kristensen 2017).

There is various guidance on conducting searching as part of systematic reviews of economic evaluations and utilities (Thielen 2016, Ara 2017, Mandrik 2021, National Institute for Health and Care Excellence 2024).

Sources to search

Some databases identify and collect economic evaluations and health economics studies to promote efficient retrieval (Glanville 2010a, Glanville 2010b, Alton 2006, Nixon 2004, Luhnen 2018, Luhnen 2019). These databases are built largely from MEDLINE and Embase, but offer a variety of value added information such as critical appraisals, results, categorisations and indexing. These databases can save time in identifying economic evaluations, but may not be comprehensive because of publication lags or geographical focus (e.g. the Cost-Effectiveness Analysis (CEA) registry). NHS EED ceased updating at the end of 2014 and is available only as a closed database. HEED is no longer available. This means that sensitive searches should also include searches of general medical databases such as MEDLINE and Embase (Glanville 2010a, Glanville 2010b, Luhnen 2018, Luhnen 2019, Royle 2003, Waffenschmidt 2010, Coyle 2012, Arber 2018). Searching Science Citation Index and conference abstracts (via websites as well as Embase) may also increase retrieval (Luhnen 2019, Coyle 2012). Pitt et al. conducted a bibliometric analysis of full economic evaluations of health interventions published in 2012-14, comparing, among other things, the sensitivity and specificity of searches in 14 databases (Pitt 2016). This study confirms that Econlit is not a high yield resource for economic evaluations and suggests that Scopus may be a useful resource to search, which may merit investigation. However, there may be a lag in practice adopting this evidence as we note that a recent survey of fifty economic models submitted to NICE (Misso 2024) reported that EconLit was still being searched for more than 60% of the models and Scopus was not reported in any.

Searching non-database sources is likely to identify further studies outside of commercial journal publications (Royle 2003). A recent survey of 50 economic models submitted to NICE (Misso 2024) reported that although 60% of models reported searches for conference abstracts only 40% reported searching grey literature such as the websites of regulatory bodies.

Most recent reviews of economic evaluations have not followed published searching approaches in detail and are also currently poorly reported (Wood 2017). Reviews should report the searches explicitly and search a range of resources (Thielen 2016, Luhnen 2019, Wood 2017). The following information sources should be considered when searching for economic evaluations and utility studies:

• Specialist economic databases for identifying economic evaluations and utility studies (CEA Registry is a live database; Paediatric Economic Database Evaluation (PEDE) is a database of pediatric economic evaluations; NHS EED closed to new records at the end of 2014) (Shemilt 2011, Sullivan 2016, Mathes 2014)

• Technology assessment databases (the Health Technology Assessment (HTA) database) (Luhnen 2018, Arber 2018)

• General medical literature databases (MEDLINE, Embase, CENTRAL) (Thielen 2016, Shemilt 2011)

• Websites of HTA agencies such as NICE, CADTH, HAS and PBAC (Thielen 2016)

  • Grey literature (Thielen 2016, Glanville2010a, Glanvlle 2010b), including but not limited to, the ISPOR Presentations database (https://www.ispor.org/heor-resources/presentations-database/search) whose interface has been recently evaluated (Cooper 2022, Cooper 2023, Lee 2024)

  • HTAi conferences (https://htai.org/annual-meetings/)

  •  ISOQoL (https://www.isoqol.org/)

  •  the RePEC economic working papers collection (National Institute for Health and Care Excellence 2024)

• Collections of utility studies including ScHARRHUD and instrument websites (Thielen 2016, Mathes 2014), as well as utility mapping collections, for example

  • the HERC Database of Mapping Studies which contains studies which map measures to EQ-5D at: http://www.herc.ox.ac.uk/downloads/herc-database-of-mapping-studies) (Dakin 2013, Dakin 2018)

  • the catalogues of EQ-5D-3L quality of life scores (Falk Hvidberg 2023)

Burden of illness studies have been collected in the HEORO burden of illness database (https://www.heoro.com/).

Identifying information to populate economic models may involve searching a range of resources including statistics, datasets, and  bibliographic databases (Glanville 2010a, Glanville 2010b, Golder 2005, Papaioannou 2013, Zechmeister-Koss 2014,De Cock 2016, Paisley 2016, Castelvecchi 2018, Sostek 2024). A recent poster describing a scoping review of health technology assessment guidance on finding model inputs notes that there is little empirical research on how to identify information for model inputs (Lister 2023). Guidance on suggested minimum searching levels for model parameters is available, although the author notes that much of the guidance has not been empirically tested (Paisley 2016). Additional suggestions for identifying utility studies include standard approaches such as checking the reference lists of eligible studies, consulting experts, carrying out citation searches and named author searching (Ara 2017) and identifying completed HTA reports (Takada 2022). One study has examined the use of routine data (typically obtained for health insurance funds or other reimbursement data sources rather than bibliographic databases) in economic evaluations and highlighted that these data may increasingly need to be included in economic evaluations (Gansen 2018). Two studies has explored challenges to incorporating environmental impacts into economic evaluations (Guirado-Fuentes 2023, Silber 2024). NICE guidance suggests that costs information might be identified from national list prices, national audit data or from the economic literature (National Institute for Health and Care Excellence 2024).

Designing search strategies

Principles of systematic review methodology should be followed for the design of search strategies to identify economic evaluations. The CHEERS 2022 statement on reporting economic evaluations recommends that authors should add "economic evaluation" to article titles to aid recall and specify the type of evaluation, but despite these recommendations search filters are still currently required in the major bibliographic databases (National Institute for Health and Care Excellence  2024, Husereau 2022a, Husereau 2022b). 

The development of sensitive subject searches within the specific economic evaluation databases is recommended to capture the population and the intervention of interest (Glanville 2010a, Glanville 2010b, Rosen 2005). An overview of methods for systematic reviews of health economic interventions suggests that a systematic search should use relevant elements of PICO combined with an economic search filter (Mathes 2014). Shemilt is more cautious still, suggesting that only intervention search terms may be required and focus can be achieved by adding the population concept (Shemilt 2011). However, there is no requirement to add an economic evaluation search filter to searches within economic evaluation databases because they are pre-filtered (Ara 2017, Nixon 2004). 

Search filters for economic studies can be considered (in combination with concepts capturing the population and/or intervention) in general bibliographic databases such as MEDLINE or Embase (Mathes 2014). Published search filters, which can be identified from the InterTASC Information Specialists' Sub-Group (ISSG) Search Filter Resource, tend to have high sensitivity but poor precision (Glanville 2009a, Glanville 2009b, McKinlay 2006). A filter validation study conducted in 2018 highlights filters with between 89.9% (2.9% precision) and 70.2% sensitivity (14.1% precision) (Harboe 2019). Cost-utility studies are often the focus of health technology assessments and a paired MEDLINE and Embase filter could be considered (Hubbard 2022). Canada's Drug Agency (CDA, formerly CADTH) offers a more precision maximizing search filter for rapid reviews (Kaunelis 201145, Moberg 2023). Search strategies to identify cost-effectiveness information may need to be adapted from those developed for searching for effectiveness studies (Droste 2011). Searching for particular economic methods may require the use of several techniques (Hinde 2011).

Searches to inform specific parameters of decision models may not be required to be as extensive and systematic as those to identify economic evaluations, as decision models are developed in an organic way, some parameters do not require the identification of comprehensive evidence and it may not be feasible to conduct extensive searches for all parameters of a model (Glanville 2010a, Golder 2005, Paisley 2016).

Health state utility values (HSUVs) are important parameters in decision models. Recent research has indicated that for NICE HTA submissions, literature reviews for utilities are typically undertaken (Castelvecchi 2018).  Searching for utilities requires specific techniques (Ara 2017, Papaioannou 2013, Brazier 2019) and the careful use of search filters can also be considered (Ara 2017). A recent case study (Lister 2024) suggests that iterative searching may be more efficient in terms of time than ‘usual practice’ (a series of sensitive database searches) or a rapid review for identifying utilities for an economic model, but recommends more research is needed. 

There are few subject headings dedicated to utilities within MeSH and EMTREE, and although general subject headings such as ‘Quality of life’ will yield relevant studies, they are likely to demonstrate poor precision (Ara 2017). Free text terms should be included in searches and three types may be helpful to include: general terms (such as QALY), instrument specific terms (such as EQ-5D) and terms describing methods of utility elicitation such as standard gamble (Arar 2017). Search filters make use of a selection of these terms which have been shown to perform well in practice (Ara 2017, Brazier 2019, Arber 2017). Arber et al. have published three validated filters for retrieving HSUVs (sensitivity maximizing; a balance of sensitivity and precision; and precision maximizing) (Arber 2017). One study recommends the use of iterative searching for utilities, following an initial scoping search, and lists factors to consider when defining the search criteria (Brazier 2019). 

Searching for cost of illness/burden of illness can make use of population search terms (perhaps taken from an accompanying effects review) (Wood 2017).

In some cases searches may also be focused, using a second filter, on specific countries or regions. Geographic filters can be identified from the ISSG Search Filter Resource (https://sites.google.com/a/york.ac.uk/issg-search-filters-resource/home/geography) and other filter collections.

Reference list

Alton 2006

Alton V, Eckerlund I, Norlund A. Health economic evaluations: how to find them. Int J Technol Assess Health Care. 2006;22(4):512-517. [Publication appraisal]

Ara 2017

Ara R, Brazier J, Peasgood T, Paisley S. The identification, review and synthesis of health state utility values from the literature. Pharmacoeconomics. 2017;35:43–55. [Publication appraisal]

Arber 2017

Arber M, Garcia S, Veale T, Edwards M, Shaw A, Glanville JM. Performance of Ovid MEDLINE search filters to identify health state utility studies. Int J Technol Assess Health Care. 2017;33(4);472-80. [Publication appraisal]

Arber 2018

Arber M, Glanville J, Isojärvi J, Baragula E, Edwards M, Shaw A, Wood H. Which databases should be used to identify studies for systematic reviews of economic evaluations? Int J Technol Assess Health Care. 2018 Jan;34(6):547-554. [Publication appraisal]

Brazier 2019

Brazier J, Ara R, Azzabi I, Busschbach J, Chevrou-Séverac H, Crawford B, Cruz L, Karnon J, Lloyd A, Paisley S, Pickard AS. Identification, review, and use of health state utilities in cost-effectiveness models: an ISPOR Good Practices for Outcomes Research Task Force report. Value Health. 2019;22(3):267–275. [Publication appraisal]

Castelvecchi 2018

Castelvecchi D. Google unveils search engine for open data. Nature. 2018;561(7722):161-162. 

Cooper 2022

Cooper C, Brown A, Court R, Schauberger U. A technical review of the ISPOR Presentations Database identified issues in the search interface and areas for future development. Int J Technol Assess Health Care. 2022;38(1):e29. [Publication appraisal]

Cooper 2023

Cooper C, Brown A, Court R, Schauberger U, Pizzi L, Willke R. A technical review of the ISPOR Presentations Database: an update on changes to the database from the authors and ISPOR. Int J Technol Assess Health Care. 2023;39(1):e8. [Publication appraisal]

Coyle 2012

Coyle KB, Trochlil K, Iversen P. MEDLINE and EMBASE for health economic literature reviews [abstract]. Value Health. 2012;15(4):A162. [Publication appraisal]

Dakin 2013

Dakin H. Review of studies mapping from quality of life or clinical measures to EQ-5D: an online database. Health Qual Life Outcomes. 2013;11(1):151. [Publication appraisal]

Dakin 2018

Dakin H, Abel L, Burns R, Yang Y. Review and critical appraisal of studies mapping from quality of life or clinical measures to EQ-5D: an online database and application of the MAPS statement. Health Qual Life Outcomes. 2018;16:202. [Publication appraisal]

De Cock 2016

De Cock E, Cosmatos I, Kirsch E. Use of databases for health resource utilizaton and cost analyses in EU-5: Results from a focused literature review [abstract]. Value in Health Conference: ISPOR 21st Annual International Meeting Research Washington, DC. Value Health. 2016;19(3):A80-A8). [Publication appraisal]

Droste 2011

Droste S, Dintsios C-M. Informationsgewinnung für gesundheitsökonomische Evaluationen im Rahmen von HTA-Berichten. Gesundheitsökonomie & Qualitätsmanagement. 2011;16(1):35-57. [Publication appraisal]

Falk Hvidberg 2023

Falk Hvidberg M, Hernandez Alava, M. Catalogues of EQ-5D-3L Health-Related Quality of Life Scores for 199 Chronic Conditions and Health Risks for Use in the UK and the USA. PharmacoEconomics 2023; 41(10):1287-1388.

Gansen 2018

Gansen, F M. Health economic evaluations based on routine data in Germany: a systematic review. BMC Health Serv Res. 2018;18(1):268. [Publication appraisal]

Glanville 2009a

Glanville J, Fleetwood K, Yellowlees A, Kaunelis D, Mensinkai S. Development and testing of search filters to identify economic evaluations in MEDLINE and EMBASE. Ottawa: Canadian Agency for Drugs and Technologies in Health; 2009. [Publication appraisal]

Glanville 2009b

Glanville J, Kaunelis D, Mensinkai S. How well do search filters perform in identifying economic evaluations in MEDLINE and EMBASE. Int J Technol Assess Health Care. 2009;25(4):522-529. [Publication appraisal]

Glanville 2010a

Glanville J, Paisley S. Searching for evidence on resource use, costs, effects and cost-effectiveness. In: Shemilt I et al (eds). Evidence based economics. Oxford:Wiley-Blackwell; 2010. [Publication appraisal]

Glanville 2010b

Glanville J, Paisley S. Identifying economic evaluations for health technology assessment. Int J Technol Assess Health Care. 2010;26(4):436-440. [Publication appraisal]

Golder 2005

Golder S, Glanville J, Ginnelly L. Populating decision-analytic models: the feasibility and efficiency of database searching for individual parameters. Int J Technol Assess Health Care.2005;21(3):305-311. [Publication appraisal]

Guirado-Fuentes 2023

Guirado-Fuentes C, Abt-Sacks A, Trujillo-Martin M et al. Main challenges of incorporating environmental impacts in the economic evaluation of health technology assessment: A scoping review. International Journal of Environmental Research and Public Health 2023;20(6):4949.

Harboe 2019

Harboe I, Desser A, Nordheim L, Glanville, J. PD85 Testing Search Filters To Retrieve Economic Evaluations In Embase. Int J Technol Assess Health Care. 2019;34(S1): 158-158. doi:10.1017/S0266462318003343. [Publication appraisal]

Hinde 2011

Hinde S, Spackman E, Claxton K, Sculpher MJ. The cost-effectiveness threshold: the results of a novel literature review method. Value Health. 2011;14:A354. [Publication appraisal]

Hubbard 2022

Hubbard W, Walsh N, Hudson T, Heath A, Dietz J, Rogers G. Development and validation of paired MEDLINE and Embase search filters for cost-utility studies. BMC Med Res Methodol. 2022;22(1):310. [Publication appraisal]

Husereau 2022a

Husereau, D, et al. Consolidated Health Economic Evaluation Reporting Standards 2022 (CHEERS 2022) statement: updated reporting guidance for health economic evaluations. BMJ. 2022;376:e067975.

Husereau 2022b

Husereau D, Drummond M, Augustovski F, de Bekker-Grob E, Briggs AH, Carswell C, et al. Consolidated Health Economic Evaluation Reporting Standards (CHEERS) 2022 explanation and elaboration: a report of the ISPOR CHEERS II Good Practices Task Force. Value Health. 2022;25(1):10-31.

Kaunelis 2011

Kaunelis D. When everything is too much: development of a CADTH narrow economic search filter. Poster presented at: 2011 CADTH Symposium. 2011 Apr 3-5; Vancouver, BC. [Publication appraisal]

Kristensen 2017

Kristensen FB, Lampe K, Wild C, Cerbo M, Goettsch W, Becla L. The HTA Core Model®—10 Years of Developing an International Framework to Share Multidimensional Value Assessment. Value in Health. 2017; 20(2): 244-250

Lampe 2009

Lampe K, Mäkelä M, Garrido MV, et al. The HTA Core Model: A novel method for producing and reporting health technology assessments. International Journal of Technology Assessment in Health Care. 2009;25(S2):9-20. doi:10.1017/S0266462309990638

Lee 2024

Lee LY, Valliant SN, Willke R, Higashi M, Pizzi L. Utility of the ISPOR Presentations Database to support the Centers for Medicare and Medicaid Services (CMS) Drug Price Negotiation (DPN): an apixaban and rivaroxaban case study [poster]. Value Health. 2024 Jun;27(6 Supplement):S348. [Publication appraisal]

Lister 2023

Lister J, Paisley S, Tappenden P, Carroll C. HTA283 A Scoping Review of Health Technology Assessment Agency (HTA) and Health Economic (HE) Modelling Guidelines for Identification of Health Economic Model Inputs [poster]. Value in Health. 2023;26(12 Supplement):S375. 

Lister 2024

Lister J, Paisley S, Carroll C, Tappenden P. Empirical Testing of Alternative Search Methods to Retrieve Utility Values for Health Economic Modelling. Pharmacoeconomics. 2024 Aug 6. doi: 10.1007/s40273-024-01414-7. [Publication appraisal]

Luhnen 2018

Luhnen M, Prediger B, Neugebauer EAM, Mathes T. Systematic reviews of economic evaluations in health technology assessment: a review of characteristics and applied methods. Int J Technol Assess Health Care. 2018;34(6):537–546. [Publication appraisal]

Luhnen 2019

Luhnen M, Prediger B, Neugebauer EA, Mathes T. Systematic reviews of health economic evaluations: A structured analysis of characteristics and methods applied. Res Syn Methods. 2019;10:2:195-206. [Publication appraisal]

Mandrik 2021

Mandrik OL, Severens JLH, Bardach A, Ghabri S, Hamel C, Mathes T, et al. Critical Appraisal of Systematic Reviews With Costs and Cost-Effectiveness Outcomes: An ISPOR Good Practices Task Force Report. Value Health. 2021;24(4):463-472.

Mathes 2014

Mathes T, Walgenbach M, Antoine SL, Pieper D, Eikermann M. Methods for systematic reviews of health economic evaluations: a systematic review, comparison, and synthesis of method literature. Med Decis Making. 2014;34(7):826-840. [Publication appraisal]

McKinlay 2006

McKinlay RJ, Wilczynski NL, Haynes RB, Hedges Team. Optimal search strategies for detecting cost and economic studies in EMBASE. BMC Health Serv Res. 2006;6:67. [Publication appraisal]

Misso 2024

Misso K, Stoniute A, Green R, Kenworthy J. HTA8 Identification of Cost-Effectiveness Evidence in UK NICE Single Technology Assessment Company Submissions: Databases, Sources and Currency of Searching. Value Health. 2024;27(6 Supplement):S245. [Publication appraisal]

Moberg 2023

Moberg K, Berg J, Eriksson T, Lundqvist M, Wiss J, Ringborg A. A comparison of two search filters for economic evaluations. The CADTH Narrow vs the NHS EED search filter [poster] Cochrane Colloquium 4 Sept. 2023. 

National Institute for Health and Care Excellence 2024

National Institute for Health and Care Excellence. Developing NICE guidelines: the manual [last updated: 17 January 2024]. London: National Institute for Health and Care Excellence; 2024.

Nixon 2004

Nixon J, Duffy S, Armstrong N, Craig D, Glanville J, Christie J, Drummond M, Kleijnen J. The usefulness of the NHS Economic Evaluation Database to researchers undertaking technology assessment reviews. Int J Technol Assess Health Care. 2004;20(3):249-257. [Publication appraisal]

Paisley 2016

Paisley S. Identification of evidence for key parameters in decision-analytic models of cost effectiveness: a description of sources and a recommended minimum search requirement. PharmacoEconomics. 2016;34(6):597-608. [Publication appraisal]

Papaioannou 2013

Papaioannou D, Brazier J, Paisley S. Systematic searching and selection of health state utility values from the literature. Value Health. 2013;16(4):686-695. [Publication appraisal]

Pitt 2016

Pitt C, Goodman C, Hanson K. Economic evaluation in global perspective: a bibliometric analysis of the recent literature. Health Econ. 2016;25 Suppl 1:9-28. [Publication appraisal]

Rosen 2005

Rosen AB, Greenberg D, Stone PW, Olchanski NV, Neumann PJ. Quality of abstracts of papers reporting original cost-effectiveness analyses. Med Decis Making. 2005;25:424-428. [Publication appraisal]

Royle 2003

Royle P, Waugh N. Literature searching for clinical and cost-effectiveness studies used in health technology assessment reports carried out for the National Institute for Clinical Excellence appraisal system. Health Technol Assess. 2003;7(34). [Publication appraisal]

Shemilt 2011

Shemilt I, Mugford M, Vale L, Craig D. Searching NHS EED and HEED to inform development of economic commentary for Cochrane intervention reviews. Oxford: Cochrane Collaboration; 2011. [Publication appraisal]

Silber 2024

Silber A, Goswami L, LaGreca E, Hamilton L, O'Hara M. EE213 Measuring healthcare's environmental impact: data availability across a spectrum of interventions and therapeutic areas. Value Health. 2024;27(6 Supplement):S96. DOI: 10.1016/j.jval.2024.03.512

Sostek 2024

Sostek K, Russell D M, Goyal N, Alrashed T, Dugall S, Noy N. Discovering Datasets on the Web Scale: Challenges and Recommendations for Google Dataset Search  Harvard Data Science Review, Special Issue 4;2024.

Sullivan 2016

Sullivan S M, Tsiplova K, Ungar W J. A scoping review of pediatric economic evaluation 1980-2014: do trends over time reflect changing priorities in evaluation methods and childhood disease?, Expert Rev Pharmacoecon Outcomes Res. 2016;16(5): 599-607. [Publication appraisal]

Takada 2022

Takada S, Narukawa M. Acceptability of manufacturer-proposed utility values for NICE cancer medicine appraisals: analysis of manufacturers’ information sources. Int J Technol Assess Health Care. 2022;38(1):e25. [Publication appraisal]

Thielen 2016

Thielen FW, Van Mastrigt G, Burgers LT, Bramer WM, Majoie H, Evers S, et al. How to prepare a systematic review of economic evaluations for clinical practice guidelines: database selection and search strategy development (part 2/3). Expert Rev Pharmacoecon Outcomes Res. 2016; 1–17. [Publication appraisal]

Waffenschmidt 2010

Waffenschmidt S, Hausner E, Engel L, Volz F, Kaiser T. Benefit of searching different databases to identify health economic evaluations in German HTA-reports. Abstract presented at: Health Technology Assessment International (HTAi) 7th Annual Meeting; 2010 June 6-9; Dublin, Ireland. Abstract T4-29. [Publication appraisal]

Wood 2017

Wood H, Arber M, Glanville JM. Systematic reviews of economic evaluations: How extensive are their searches? Int J Technol Assess Health Care. 2017; 27:1-7. [Publication appraisal]

Zechmeister-Koss 2014

Zechmeister-Koss I, Schnell-Inderst P, Zauner G. Appropriate evidence sources for populating decision analytic models within health technology assessment (HTA): a systematic review of HTA manuals and health economic guidelines. Med Decis Making. 2014;34(3):288-299. [Publication appraisal]