To cluster or not to cluster - that is the question?
by Yoav Ben-Shlomo (Prof. of Clinical Epidemiology, Population Health Sciences, University of Bristol.)
One of the joys of being a researcher is that you never stop learning new things.
We are all familiar with different trial designs whereby a participant can be randomised at an individual level (iRCT) or at a group or cluster level (cRCT). Within the context of Health Services and Delivery Research (HS&DR), we need evidence that an intervention improves the quality, safety, accessibility and organisation of health and/or social care services. For example, we may wish to evaluate whether a teleclinic for monitoring chronic kidney disease was as effective but cheaper and more convenient than an out-patient department (OPD) based service. Individuals within a renal unit could be randomised to either teleclinic or OPD (iRCT). Alternatively, renal units could be randomised to teleclinic or OPD (cRCT).
But what if in an individual based trial, the intervention arm itself has groups or clusters?
A cardiac rehabilitation trial might recruit 10 centres and within each centre, eligible participants were randomised to routine care or a new group based exercise-programme run by two different facilitators at that centre. In this case randomization is done at an individual not group level but participants who receive the intervention are naturally clustered within different groups (both within and between centres) and all participants (intervention and control) are clustered within hospital centre.
Should you or should you not ignore the clustering? If you do want to account for the clustering, what do you do about the control participants, unlike the intervention participants, who belong to a hospital centre but do not have a group cluster as they do not receive the intervention.
Thankfully a paper by Laura Flight and colleagues1 has addressed this issue, at least for a continuous outcome measure, for example daily activity levels measured by an accelerometer.
Their paper, which is not overly complex in terms of statistics, explains the pros and cons of several approaches and looks at four case studies. One could forget about the clustering but this is technically wrong as it firstly ignores the actual study design and more importantly assumes all the observations are independent regardless of cluster. This will mean that the standard errors and p-values are smaller than they should be (more likely to find a “significant” results – type I error) but is dependent on the degree of clustering. If the clustering is very weak the results are unlikely to be very different.
However, if one wants to retain the clustering in the analysis we have to consider what to do with the control arm. They discuss various options for a “partial clustered model”:
(i) Assume all controls belong to one cluster or in this example all controls in each hospital centre belong to a single cluster
(ii) Assume each control participant acts as their own cluster (n=1) within centre or
(iii) Artificially create clusters and randomly allocate control participants to these within hospital centres. Pragmatically the number of control clusters is usually the same as the number of intervention clusters.
Using a random effects model, (they do not discuss the fixed effects model) they describe how one can do a partially clustered model and then in addition take into account that the outcome in the intervention may be more similar in the treatment than control arm by nature of having a group based intervention. Hence one must model heteroskedastic individual level errors. If this is getting too technical for you, you should certainly be seeking the help of a statistician.
They then test out these different approaches using 4 case studies. In most cases the results are almost identical regardless of the analytical method. This reflects that the intra-class correlations (ICC), a measure of the clustering, were small in each case study (<=0.04) but this cannot be assumed to always be true and if nothing else the researcher should look at this first before undertaking these more complex approaches and report this in any publications. Finally, one needs to consider this design from the outset any sample size calculations will be too small if you ignore the clustering and hence your study may be under-powered.
The authors conclude that ignoring clustering in such a design should be avoided though there is clearly need for further research on the simplest model that captures the design but does not over complicate the model. HSDR researchers my experience these hybrid scenarios more often than conventional drug (individual-based) or public health trials (cluster-based) and need to consider these analytical issues when producing their statistical analysis plan.