Where we have cluster-adjusted results from trials (summary statistics), under what circumstances can we include them in a standardised mean difference (SMD) meta-analysis and which calculations are necessary?

For example, we have a statistical analysis that used generalised linear mixed models, with care home as a random factor, and time as a repeated measure, and reports mean and standard error (SE) for each intervention. We know the number of individuals in the intervention groups and the number of clusters.

If this was not cluster-randomised, I would calculate the SD for each group = SE x √N and then the SMD using the standard formulae for Hedges’ g, as implemented in RevMan. For calculating the SD, pooled SD, SMD and SE{SMD} I would be using values of N = n1 + n2.

For cluster-randomised results, is it appropriate to be using the raw number of individuals in the intervention groups, a cluster-adjusted number of individuals using the design effect, i.e. Nadj = N / (1 + (M-1)ICC), or is it not appropriate to calculate SMD from these data?

Similarly, given an adjusted MD(SE) is it appropriate to calculate SD using the standard formula of SD = SE / √(1/n1 + 1/n2) using raw values for n?

For good measure, given unadjusted means, SDs and ns for a cluster trial, it's presumably okay to adjust the ns using the design effect and then calculate SMD and SE{SMD}. [edit: no, it isn't, see references provided by @Wolfgang]


  • $\begingroup$ Dear Tom, I am not sure I can follow you. In general terms, if you have only aggregate data, I recommend using multivariate meta-analysis in Stata or R. If you have subject-level data, then any modeling tool in which you can add study ID as a random effect factor should make sense, as long as the other features are correctly modeled. $\endgroup$ Jun 29, 2019 at 8:59
  • $\begingroup$ @Joe_74 I only have aggregate data. I'm not sure using multivariate meta-analysis would solve the problem of calculating a common effect size among different scales when the data comes from cluster trials. $\endgroup$ Jul 4, 2019 at 13:25

1 Answer 1


Not a direct answer, but you might want to dig into these articles:

Hedges, L. V. (2007). Effect sizes in cluster-randomized designs. Journal of Educational and Behavioral Statistics, 32(4), 341-370.

Hedges, L. V. (2011). Effect sizes in three-level cluster-randomized experiments. Journal of Educational and Behavioral Statistics, 36(3), 346-380.

Hedges, L. V., & Citkowicz, M. (2015). Estimating effect size when there is clustering in one treatment group. Behavior Research Methods, 47(4), 1295-1308.

White, I. R., & Thomas, J. (2005). Standardized mean differences in individually-randomized and cluster-randomized trials, with applications to meta-analysis. Clinical Trials, 2(2), 141-151.

  • 1
    $\begingroup$ Thanks. For my purposes White and Thomas 2005 and Hedges 2007 seem most useful. I have also found Walwyn, R., and Roberts, C. ( 2017) Meta‐analysis of standardised mean differences from randomised trials with treatment‐related clustering associated with care providers. Statist. Med., 36: 1043– 1067. doi: 10.1002/sim.7186. which gives a more general answer. I am not finding any of these easy to follow but am still persevering and will provide an answer if I can! $\endgroup$ Jul 4, 2019 at 13:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.