I'm trying to find the minimum detectable effect size (MDES) given my sample, alpha (.05), and desired power (90%) in a linear mixed model setting. I'm using the simr
package for a simulation-based approach. What I did is changing the original effect size to a series of hypothetical effect sizes and find the minimum effect size that has a 90% chance of producing a significant result. Below is a toy code:
library(lmerTest)
library(simr)
# fit the model
model <- lmer(Reaction ~ Days + (Days | Subject), sleepstudy)
summary(model)
Model's Fixed effect:
Fixed effects:
Estimate Std. Error df t value Pr(>|t|)
(Intercept) 251.405 6.825 17.000 36.838 < 2e-16 ***
Days 10.467 1.546 17.000 6.771 3.26e-06 ***
Here is the code for minimum detectable effect size:
pwr <- NA
# define a set of reasonable effect sizes
es <- seq(0, 10, 2)
# loop through the effect sizes
for (i in 1:length(es)) {
# replace the original effect size with new one
fixef(model)['Days'] = es[i]
# run simulation to obtain power estimate
pwr.summary <- summary(powerSim(
model,
test = fixed('Days', "t"),
nsim = 100,
progress = T
))
# store output
pwr[i] <- as.numeric(pwr.summary)[3]
}
# display results
cbind("Unstandardized Regression Coefficient" = es,
Power = pwr)
Output:
Unstandardized Regression Coefficient Power
[1,] 0 0.09
[2,] 2 0.24
[3,] 4 0.60
[4,] 6 0.99
[5,] 8 1.00
[6,] 10 1.00
My questions:
(1) Is this the right way to find the MDES?
(2) I have some trouble making sense of the output. Can I say the following: because the estimated power when the effect = 6 is 99%, and because the actual model has an estimate of 10.47, then the study is sufficiently powered? Conversely, imagine that if the actual estimate was 3.0, then can I say the study is insufficiently powered?
simr
but by default it didn't used to use the Kenward-Roger or Satterthwaite approximations, which have been shown to be preferable: Luke, S. G. (2017). Evaluating significance in linear mixed-effects models in R. Behavior research methods, 49(4), 1494-1502. $\endgroup$