Kreis 2014

Appraisal of: Kreis, J, Panteli, D. How health technology assessment agencies address the issue of unpublished data. Int J Tech Assess Health Care 2014;30(1):34-43.

Reviewer(s): 

Carolyn Spry

Melissa Walter

Full Reference: 

Kreis, J, Panteli, D. How health technology assessment agencies address the issue of unpublished data. Int J Tech Assess Health Care 2014;30(1):34-43.

Short description: 

The study provided an overview of policies and practices of eighteen HTA agencies in searching, including, and reporting unpublished data in their assessments. Methods papers detailing efforts to include unpublished data and HTA reports prepared by these agencies were examined.

The results showed that the methods papers of 18 agencies varied greatly in comprehensiveness, currentness, criteria for including unpublished data and guidance on reporting it. The study authors looked at 73 HTA reports that described efforts to include unpublished data. The main efforts to include unpublished data included requests to industry, searches of trial registries and websites (mostly FDA), and hand searching conference proceedings.

Limitations stated by the author(s): 

Approximately a quarter of nondrug interventions assessed in the HTA reports of our sample did not involve the application of a medical device so that manufacturers could not be contacted. We did not systematically evaluate whether in these cases other possible owners of data, e.g., research groups at universities, were contacted. Selective publication of studies, however, may be equally (or even more) prevalent in academic research, especially with regard to study designs other than clinical trials. A challenge in the evaluation of interventions not commercially owned is that it is less clear who may have carried out studies or who may be in possession of the data, and the diversity of relevant parties in this field may make requests difficult. In addition, our analysis, which was exploratory in nature, was restricted to information available in publicly available documents, and, despite our efforts, we may have failed to identify all relevant documents. Some methods papers may also not reflect current policies, as indicated by some of the respondents to our inquiries. In the absence of other information available, we treated methods papers equally, regardless of considerable differences with regard to their comprehensiveness and their currentness. As the consideration of reporting bias in HTA has changed over time, differences between methods papers in this regard may also be related to the time at which they were written. Inevitably, our study does not reflect current activities or discussions at the HTA agencies that are yet unconsidered in official documents. Thus, future research could strive for amore extensive participatory approach to expand our knowledge on strategies identified in our study and agencies’ experiences.

Limitations stated by the reviewer(s): 

No additional limitations were detected by the reviewers.

Study Type: 

Single study

Related Chapters: 

Tags: 

• Grey literature

• Conference abstracts