7
$\begingroup$

I have conducted a within-subject experiment. The DV of the experiment is a categorical variable (0 or 1) and the IV has three levels (1 = none, 2 = weak, 3 = strong) Since the DV is binary, I've built a logistic mixed-effects regression model. Here is the code I ran below:

model <- glmer(DV ~ IV + (1|subjects) + (1|items), data=df, family=binomial("logit"))

The model has been converged as below:

Generalized linear mixed model fit by maximum likelihood (Laplace Approximation) ['glmerMod']
Family: binomial  ( logit )
Formula: DV ~ IV + (1 | subjects) + (1 | items)
Data: df

AIC      BIC   logLik deviance df.resid 
1370.3   1398.4   -680.2   1360.3     2035 

Scaled residuals: 
   Min      1Q  Median      3Q     Max 
 -9.0454 -0.2655  0.1214  0.2882  7.7130 

 Random effects:
   Groups   Name        Variance Std.Dev.
 items    (Intercept) 1.9754   1.4055  
 subjects (Intercept) 0.4548   0.6744  
 Number of obs: 2040, groups:  items, 60; subjects, 34

 Fixed effects:
              Estimate Std. Error z value             Pr(>|z|)    
   (Intercept)  -3.1913     0.3899  -8.185 0.000000000000000272 ***
   type.strong    6.1598     0.5401  11.405 < 0.0000000000000002 ***
   type.weak      4.2876     0.5046   8.498 < 0.0000000000000002 ***
   ---
   Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

 Correlation of Fixed Effects:
   (Intr) typstr
 typestrong -0.682       
 typeweak   -0.714  0.541

Apparently, my model shows that the IV has a significant effect. However, when I convert each coefficient of the fixed effects into odds ratio, the outcome is too high in my view as you can see below.

exp(6.1598) # 473.3334
exp(4.2876) # 72.79156

Originally, I expected those coefficients I've mentioned to be negative (or at least much less than 6.1598 and 4.2876). But I got strong doubt on whether I have taken a proper procedure when the model showed those results. So, here are my questions.

  1. Is it okay that the direction of the fixed effects is opposite to the intercept in my model?

  2. Most importantly, is the odds ratio I've calculated usable? i.e. can my IV function as a proper predictor?

  3. Or have I missed or done something wrong?

It's my first time to run a logistic mixed-effects model of which IV has three levels. Any help would be enormously appreciated, thanks in advance!

$\endgroup$
  • 2
    $\begingroup$ Do the results change if you set nAGQ to values higher than the default of 1 ? Also, what if you use a different function like mixed_model from package GLMMAdaptive with nAGQ =11? $\endgroup$ – Robert Long Jan 22 at 7:42
  • $\begingroup$ The intercept is the log-odds of the event for the those in the none group (so being negative it is a protective effect), while the estimates for the other 2 groups are the log odds of the event in each of those groups compared to the none group (so they are risk factors). Does that make more sense ? $\endgroup$ – Robert Long Jan 22 at 7:55
11
$\begingroup$

Is it okay that the direction of the fixed effects is opposite to the intercept in my model?

Yes. The interpretation of the intercept is that it is the log-odds of the event (DV = 1) for the those in the none group (so, being negative, it is a protective effect), while the estimates for the other 2 groups are the log-odds of the event in each of those groups compared to the none group (so being positive, they are risk factors). So the log-odds for the event for those in the weak group are 4.2876 higher than in the none group, and the log-odds for the the event for those in the strong group are 6.1598 higher than for those in the none group.

Most importantly, is the odds ratio I've calculated usable? i.e. can my IV function as a proper predictor?

Yes, but note that the fixed effects are conditional on the random effects. This means in your case, the log odds - or odds ratios if you exponentiate them - are for the same subject and the same item, rather than being averaged across all subjects and items. If you want the averaged estimates then you could get those using the GLMMAdaptive package

Or have I missed or done something wrong?

Without more information about your study design it is difficult to say, but if you are worried about the results note that by default, glmer uses the Laplace approximation which means only 1 point per axis for evaluating the adaptive Gauss-Hermite approximation to the log-likelihood, and this can produce biased results with a binary outcome. Try setting nAGQ = 2 or higher. You could also try using the mixed_model function in the GLMMAdaptive package which is specifically written for adaptive quadrature in generalized linear mixed models.

$\endgroup$
  • $\begingroup$ Thank you so much with the detailed answers! I'm quite relieved for now. In terms of setting the value of nAGQ, when I ran my model set with nAGQ = 2, R says 'nAGQ > 1 is only available for models with a single, scalar random-effects term'. If I understood it well, to run the model with nAGQ > 1, the model should have a single random effect, right? But as you can see in my original post, my model has two random effects: subjects and items. I want to build a model with more than a single random factor. $\endgroup$ – Kihyo Jan 23 at 2:47
  • $\begingroup$ So, as you've suggested, I tried to use the 'mixed-model' function in the GLMMAdaptive package (Thanks for introducing this one, by the way) and got some sense of how to use it with a single random factor. But I can't figure out a way to write a code that'll build a model with multiple random effects. If you don't mind, would you please let me know how to build that sort of model? $\endgroup$ – Kihyo Jan 23 at 2:48
  • 1
    $\begingroup$ Ahh sorry, it seems GLMMAdaptive only supports single random effects. I think you could use either glmmADMB or glmmTMB $\endgroup$ – Robert Long Jan 23 at 3:19
  • $\begingroup$ I've tried glmmTMB and it perfectly worked with multiple random effects. The result has not been changed dramatically compared to the one I've got with glmer, but I think it would be better to use glmmTMB than doing glmer when I get a chance to build generalized linear mixed models next time. I have had no idea about nAGQ at all before you mentioned it. Again, thanks for your help. I really appreciate it! $\endgroup$ – Kihyo Jan 23 at 4:08
  • 1
    $\begingroup$ @Kihyo You're welcome. Glad I could help! If it answers your question please consider marking it as the accepted answer. $\endgroup$ – Robert Long Jan 23 at 7:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.