I am wondering if there are any methods for calculating sample size in mixed models? I'm using lmer in R to fit the models (I have random slopes and intercepts).

  • 3
    $\begingroup$ Simulation is always an option - i.e. simulate data under a particular alternative hypothesis and sample size and re-fit the model many times to see how often you reject the null hypothesis of interest. From my experience this is quite (computer) time consuming as it takes at least a few seconds for each model fit. $\endgroup$ – Macro Jan 29 '13 at 13:55

The longpower package implements the sample size calculations in Liu and Liang (1997) and Diggle et al (2002). The documentation has example code. Here's one, using the lmmpower() function:

> require(longpower)
> require(lme4)
> fm1 <- lmer(Reaction ~ Days + (Days|Subject), sleepstudy) 
> lmmpower(fm1, pct.change = 0.30, t = seq(0,9,1), power = 0.80)

     Power for longitudinal linear model with random slope (Edland, 2009) 

              n = 68.46972
          delta = 3.140186
         sig2.s = 35.07153
         sig2.e = 654.941
      sig.level = 0.05
              t = 0, 1, 2, 3, 4, 5, 6, 7, 8, 9
          power = 0.8
    alternative = two.sided
       delta.CI = 2.231288, 4.049084
           Days = 10.46729
        Days CI = 7.437625, 13.496947
           n.CI = 41.18089, 135.61202

Also check the liu.liang.linear.power() which "performs the sample size calculation for a linear mixed model"

Liu, G., & Liang, K. Y. (1997). Sample size calculations for studies with correlated observations. Biometrics, 53(3), 937-47.

Diggle PJ, Heagerty PJ, Liang K, Zeger SL. Analysis of longitudinal data. Second Edition. Oxford. Statistical Science Serires. 2002

Edit: Another way is to "correct" for the effect of clustering. In an ordinary linear model each observation is independent, but in the presence of clustering observations are not independent which can be thought of as having fewer independent observations - the effective sample size is smaller. This loss of effectiveness is known as the design effect :

$$ DE = 1 +(m-1)\rho$$ where $m$ is the average cluster size and $\rho$ is the intraclass correlation coefficient (variance partition coefficient). So the sample size obtained through a calculation that ignores clustering is inflated by $DE$ to obtain a sample size that allows for clustering.

| cite | improve this answer | |
  • 3
    $\begingroup$ This design effect is only relevant for the overall linear statistics (means, totals). For regression coefficients, the DEFF is more like $${\rm DEFF} = 1 + (m-1) \rho_x \rho_\epsilon,$$ where $\rho_x$ is the ICC of the regressor and $\rho_\epsilon$ is the ICC of the error term (composite error = cluster random effect + observation specific effect). Due to the product of the correlations that tend to be small, this DEFF is also smallish. $\endgroup$ – StasK Jan 29 '13 at 15:23
  • 1
    $\begingroup$ Can you point me toward a citation for this formula? $\endgroup$ – Joshua Rosenberg Jul 16 '17 at 16:41

For anything beyond the simple 2 sample tests I prefer to use simulation for sample size or power studies. With prepackaged routines you can sometimes see large differences between the results from the programs based on the assumptions that they are making (and you may not be able to find out what those assumptions are, let alone if they are reasonble for your study). With simulation you control all the assumptions.

Here is a link to an example:

| cite | improve this answer | |
  • $\begingroup$ Just wondering, does this also work for GLMER models? $\endgroup$ – Charlie Glez Apr 5 '16 at 18:56
  • 1
    $\begingroup$ @CarlosGlez, yes, this works for any model where you can simulate data and analyze it. I have done this for GLMER models. $\endgroup$ – Greg Snow Apr 6 '16 at 15:28
  • $\begingroup$ Well said, and I'll add that in addition to "controlling assumptions", you can also ask "what if" questions, break these assumptions, and determine some practical sense of robustness, e.g. whether non-normal random effects really ruin efficiency. $\endgroup$ – AdamO Apr 26 '16 at 19:48

The simr package uses simulation to estimate power fairly flexibly in linear and generalised linear mixed models.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.