1
$\begingroup$

I am trying to model the effect of 3 continuous predictors on a binary choice outcome. These data come from an experiment in which multiple subjects were ran in a task containing multiple trials (hence the random intercept of subject, and the random slopes for each predictor). Suppose I have these three nested linear mixed models (using lme4 package in R):

model.1 <- glmer(choice ~ 1 + beta1 + beta2 + (1 + beta1 + beta2|subject),
                         data=data, family=binomial(link="probit"))

model.2 <- glmer(choice ~ 1 + beta1 + beta2 + criticalbeta + (1 + beta1 + beta2 + criticalbeta|subject),
                         data=data, family=binomial(link="probit"))

model.3 <- glmer(choice ~ 1 + (beta1 + beta2) * criticalbeta + (1 + beta1 + beta2 + criticalbeta|subject),
                         data=data, family=binomial(link="probit"))

What I am interested in answering is not so much whether beta1, beta2, and criticalbeta are significant, but whether the addition of the criticalbeta term as an additive or multiplicative parameter, better fits the data when modeling the outcome choice. Additionally, it could be that neither the additive model or multiplicative model fit the data better, in which case I would conclude that the addition of the criticalbeta was not necessary.

What I have been doing to compare these models, since the terms are nested, is running them through the anova() like so:

anova(model.1, model.2, model.3)

This gives me the output (similar to):

Data: data

Models:
model.1: choice ~ 1 + beta1 + beta2 + (1 + beta1 + beta2|subject)  
model.2: choice ~ 1 + beta1 + beta2 + criticalbeta + (1 + beta1 + beta2 + criticalbeta|subject)        
model.3: choice ~ 1 + (beta1 + beta2) * criticalbeta + (1 + beta1 + beta2 + criticalbeta|subject)

         Df   AIC   BIC  logLik deviance   Chisq Chi Df Pr(>Chisq)    
model.1  46 14516 14878 -7211.8    14424                              
model.2  67 14455 14983 -7160.7    14321 102.332     21  1.114e-12 ***
model.3  83 14467 15120 -7150.3    14301  20.744     16     0.1886    
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

While I believe this is the easiest way to compare the model that best fits the data, I am a little stuck on how I would go about constructing a power analysis for this model comparison. I need to be able to justify that I can collect enough data (i.e. enough subjects, with enough trials) to have >95% power to detect an effect.

I have found resources online for how to conduct a power analysis for each model, but can anyone point me to resources on (or explain, possibly) how one would go about setting up a power analysis for a model comparison?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.