Sterne JAC, et al. RoB 2: a revised tool for assessing risk of bias in randomised trials. BMJ. 2019 Aug 28;366:l4898. (8-page paper)

• Summary: Describes an updated version of the Cochrane Collaboration’s tool (2011) for assessing risk of bias in randomized trials. This tool, which concisely summarizes several trial-related biases, can be used to better understand biases more generally, allowing a researcher - embarking on the design of a new trial - to mitigate such biases before they go on to impact the validity of the published study. (i.e., Having the ability to critically assess a published trial’s risk of bias is useful when designing a trial as well.) Version 2 of the Cochrane risk-of-bias assessment tool describes: (a) bias domains, (b) signaling questions, (c) response options, and (d) risk-of-bias judgments. The domains include: (1) bias arising from the randomization process, (2) bias due to deviations from intended interventions, (3) bias due to missing outcome data, (4) bias in measurement of the outcome, and (5) bias in selection of the reported result. * For those at the design stage of a trial: to assess whether your approaches can be refined to mitigate biases before they arise, it may be useful to subject your draft trial protocol to risk of bias assessments.

Part 2: An overview of RoB 2 (May 13, 2020 via “Cochrane Training”): 19-min video (slides 17-28).

• Summary: While embarking on the design of a (pragmatic) clinical trial is not the same as assessing the risk-of-bias in an already-published trial (using the Cochrane tool for assessing risk of bias - RoB 2), this video succinctly overviews biases that can arise at various stages: i.e., (1) bias arising from the randomization process, (2) bias due to deviations from intended interventions, (3) bias due to missing outcome data, (4) bias in measurement of the outcome, and (5) bias in selection of the reported result. Ultimately, identifying your effect of interest and questioning (using “signaling questions”) why it may be difficult to estimate that effect (i.e., what biases may arise during the design, implementation, and management of the study) is a useful exercise; the RoB 2 tool has mapped signaling questions to specific response options, and those options are associated with various risk-of-bias judgments (i.e., lower risk of bias vs. higher risk of bias). An introduction to adapted tools that address issues in cluster randomized trials and crossover trials is provided, as well.

Phillips MR, et al. Risk of bias: why measure it, and how? Eye (Lond). 2022 Feb;36(2):346-348. (3-page editorial)

• Summary: Bias exists “when a component of the design or execution of a study has systematic impacts on the results of the study that deviate from the truth.” This paper briefly overviews selection bias, performance bias, detection bias, attrition bias, and reporting bias. It is argued that not all randomized trials are the same, and that careful consideration needs to be taken when determining if trial results are worthy of informing decision making (e.g., changing the management of future patients).

What on earth does bias mean in a pragmatic trial - Prof Sandra Eldridge (December 9, 2021 via "HRB TMRN"): 15-min video.

• Summary:  Prof. Sandra Eldridge defines bias and goes on to describe this topic in the context of pragmatic trials. i.e., The goal of a pragmatic trial differs from that of a more explanatory trial, and how bias manifests in these settings may differ. The Cochrane RoB 2 and PRECIS-2 tools are discussed, as well as PICOS (Population, Intervention, Control, Outcome, Setting); both tools, and the PICOS framework, when used together, can clarify potential sources of bias in a pragmatic trial. It is concluded that: bias is about the distortion of an (estimated) effect, bias assessment requires knowing what the goal (aim/target) is of a given trial, biases relate to both internal and external validity, and that some biases can be best judged using the RoB 2 tool, while others relate more to domains captured in the PRECIS-2 tool. Ultimately, potential biases within trials need to be judged on a study-by-study basis.

Higgins JPT, et al. Chapter 8: Assessing risk of bias in a randomized trial [last updated October 2019]. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.5. Cochrane, 2024. (book chapter)

• Summary: Reiterates and expands upon the process related to risk of bias assessments using the RoB 2 tool. There are additional resources specific to cluster randomized trials, stepped-wedge trials, crossover trials, and factorial trials in Chapter 23: Including variants on randomized trials (Higgins JPT, Eldridge S, Li T [last updated: October 2019]). * For those at the design stage of a trial: to assess whether your approaches can be refined to mitigate biases before they arise, it may be useful to subject your draft trial protocol to risk of bias assessments.

Part 1: An overview of how bias can arise during the randomization process (June 29, 2020 via “Cochrane Training”): 9-min video (slides 1-13).

• Summary: Describes bias domain (1): bias arising from the randomization process. The signaling questions for this domain are: 1.1 Was the allocation sequence random? 1.2 Was the allocation sequence concealed until participants were enrolled and assigned to interventions? 1.3 Did baseline differences between intervention groups suggest a problem with the randomization process?

Part 1: Deviations from intended intervention and the role of blinding (July 21, 2020 via “Cochrane Training”): 14-min video (slides 1-11).

• Summary: Describes bias domain (2): bias due to deviations from intended interventions. The signaling questions for this domain are: 2.1 Were participants aware of their assigned intervention during the trial? 2.2 Were the people delivering the interventions aware of participants’ assigned intervention during the trial? 2.3 If applicable: Were there deviations from the intended intervention that arose because of the trial context? 2.4 If applicable: Were these deviations likely to have affected the outcome? 2.5 If applicable: Were these deviations from intended intervention balanced between groups? 2.6 Was an appropriate analysis used to estimate the effect of assignment to intervention? 2.7 If applicable: Was there potential for a substantial impact (on the result) of the failure to analyze participants in the group to which they were randomized?

Part 1: Missing outcome data and when they lead to bias (August 5, 2020 via “Cochrane Training”): 25-min video (slides 1-14).

• Summary: Describes bias domain (3): bias due to missing outcome data. The signaling questions for this domain are: 3.1 Were data for this outcome available for all, or nearly all, participants randomized? 3.2 If applicable: Is there evidence that the result was not biased by missing outcome data? 3.3 If applicable: Could missingness in the outcome depend on its true value? 3.4 If applicable: Is it likely that missingness in the outcome depended on its true value?

Part 1: Bias mechanisms and empirical evidence (September 18, 2020 via “Cochrane Training”): 15-min video (slides 1-14).

• Summary: Describes bias domain (4): bias in measurement of the outcome. The signaling questions for this domain are: 4.1 Was the method of measuring the outcome inappropriate? 4.2 Could measurement or ascertainment of the outcome have differed between intervention groups? 4.3 If applicable: Were outcome assessors aware of the intervention received by study participants? 4.4 If applicable: Could assessment of the outcome have been influenced by knowledge of intervention received? 4.5 If applicable: Is it likely that assessment of the outcome was influenced by knowledge of intervention received?

Part 1: Introduction; selective reporting vs selective non-reporting (October 16, 2020 via “Cochrane Training”): 8-min video (slides 1-8).

• Summary: Describes bias domain (5): bias in selection of the reported result. The signaling questions for this domain are: 5.1 Were the data that produced this result analyzed in accordance with a prespecified analysis plan that was finalized before unblinded outcome data were available for analysis? Is the numerical result being assessed likely to have been selected, on the basis of the results, from: 5.2 ... multiple eligible outcome measurements (e.g., scales, definitions, time points) within the outcome domain? 5.3 ... multiple eligible analyses of the data?

Mansournia MA, et al. Biases in Randomized Trials: A Conversation Between Trialists and Epidemiologists. Epidemiology. 2017 Jan;28(1):54-59. (6-page paper)

• Summary: Different terminology may be used to describe biases in randomized trials and observational studies; the language used in Cochrane trial-related bias resources (e.g., selection bias, performance bias, detection bias, attrition bias, and reporting bias) may differ slightly from the observational research literature. This paper provides a “translation” of key bias-related terms, emphasizing that awareness of various terminologies will enhance communication between trialists and epidemiologists focused on observational research.

Boyd AD, et al. Potential bias and lack of generalizability in electronic health record data: reflections on health equity from the National Institutes of Health Pragmatic Trials Collaboratory. J Am Med Inform Assoc. 2023 Aug 18;30(9):1561-1566. (6-page paper)

• Summary: Pragmatic trials that leverage routinely collected health data (including electronic health records) promote efficiencies that increase the amount of robust generalizable research. However, as the number of pragmatic trials using routinely collected data increases, so does the risk that research will become more susceptible to biases due to differences in data capture and access to care for different subsets of the population. This paper summarizes 3 challenges: (1) incomplete/variable capture of data on social determinants of health, (2) lack of representation of vulnerable populations that do not access or receive care, and (3) data loss due to variable use of technology; recommendations for how to mitigate such biases are provided.

Kahan BC, et al. Risk of selection bias in randomised trials. Trials. 2015 Sep 10;16:405. (7-page paper)

• Summary: Selection bias occurs when those in charge of the recruitment of patients selectively enrol patients based on what the next treatment allocation is likely to be. This paper summarizes a review of published trials that assessed whether investigators were taking adequate steps to reduce the risk of selection bias; it was found that the risk of selection bias could not be ascertained for most trials due to poor reporting (i.e., a lack of detail on the randomization procedure used in the study).