Arevalo 2023

Appraisal of: Arevalo-Rodriguez I, Baxter S, Steingart KR et al. How to develop rapid reviews of diagnostic tests according to experts: A qualitative exploration of researcher views.
Reviewer(s): 

Julie Glanville

Full Reference: 

Arevalo-Rodriguez I, Baxter S, Steingart KR et al. How to develop rapid reviews of diagnostic tests according to experts: A qualitative exploration of researcher views. Cochrane Evidence Synthesis and Methods. 2023;1:e12006. DOI: 10.1002/cesm.12006

Short description: 

Methods

The study aimed to explore experts’ perceptions of methods for developing rapid reviews (RRs) of diagnostic tests. The research used online semi-structured interviews. A purposive sampling strategy was used to select experts and users of diagnostic evidence syntheses. The sample aimed for diversity in geographical region, country, organization, and role (e.g., methodologists, biostatisticians, information specialists, RR guidance developers, and knowledge users). 23 interviewees were invited by email and 16 (70%) responded and participated in Zoom interviews conducted between July and Dec 2021. Interviews were transcribed and checked for accuracy.

A coding framework using NVIVO software was developed from a literature review and refined iteratively using thematic analysis. Each transcript was coded according to themes. Trustworthiness was enhanced by independent coding, sequential interviews (allowing for topic confirmation), and highlighting differing viewpoints.

The study was reported according to Consolidated Criteria for Reporting Qualitative Research (COREQ) and the Standards for Reporting Qualitative Research (SRQR).

.

Results relating to the search aspects of rapid reviews

Experts differed about limiting the number of electronic databases searched. Some emphasized the importance of identifying the right sources rather than the sheer number of databases, while others believed that searching several databases was always necessary.

Many participants acknowledged that, under time constraints, compromises are inevitable. Limiting searches may result in missing some studies, but most experts felt this would not dramatically affect the overall conclusions if the main sources were covered.

The appropriateness of limiting searches depends on the specific topic and research question. For example, if a diagnostic test is new or only recently available, limiting the search by date is considered appropriate.

Some experts advocated for using methodological filters to reduce the number of records retrieved, seeing it as a pragmatic approach. Others considered filters risky or inaccurate, potentially missing relevant studies.

Grey Literature and Conference Abstracts: Decisions about searching these sources were often made on a case-by-case basis, depending on available time and resources.

Some participants supported limiting searches to English for practical reasons, while others argued against it, citing the availability of translation tools and the risk of bias from excluding non-English studies.

Limiting searches by date was considered appropriate, especially when the diagnostic test of interest was only available after a certain date.

Note that the paper also addresses other non-search related methods.

Limitations stated by the author(s): 

The key limits seem to centre on generalisability. The authors noted that they had a small sample size and could not recruit as many interviewees as they had planned. This may have led to selection bias, and some perspectives or country-specific practices might not have been captured.

The lead interviewer was also a methodologist involved in diagnostic evidence synthesis. Her interpretations during interviews and analysis could be subject to bias.

Many interviewees' opinions were primarily based on their experiences synthesizing evidence about tests for diagnosing severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) during the COVID-19 pandemic. This may limit the generalizability of findings to other diagnostic test scenarios or non-pandemic settings.

The research reflects experts’ perceptions, rather than empirical evaluations of rapid review methods.

Limitations stated by the reviewer(s): 

The findings rely on self-reported practices and perceptions, which may not always reflect actual behaviours or real world outcomes.

There is the potential for bias because the interviews were conducted in English.

SuRe Info keywords (pre-defined tags):

Diagnostic test accuracy studies

Study Type: 

Single qualitative study

Related Chapters: 

Tags: 

• Search limits

• Searching

• Rapid reviews