# Odds ratio confidence intervals and p-values suggest different conclusions in a binary logistic mixed-effects model (glmer)

I am running a generalized linear mixed-effects model in R using the glmer function of lme4. The outcome variable is trial-level accuracy in a task (incorrect trials are 0, correct trials are 1), and I have 3, trial-level predictors:

• A, a binary categorical variable
• B, another binary categorical variable
• C, a categorical variable with 3 levels

(All variables, including accuracy are factors.)

Here is the model I'm running:

    acc_model <- glmer(accuracy ~ 1 + A*B*C + (1|subid), data = acc_data, family = binomial,
control = glmerControl(optimizer ="bobyqa"))
#subid simply refers to subject IDs as trials are nested within participants
#I use bobyqa because the model failed to converge with the default optimizer


Then I use the Anova() function from the "car" package to see if the terms are significant.

library(car)
Anova(acc_model)


This has worked nicely for me in the past (anova() does not give p-values for glmer models), and the results seem sensible in this case too:

Analysis of Deviance Table (Type II Wald chisquare tests)

Response: acc
Chisq Df Pr(>Chisq)
A                           10.0468  1   0.001526 **
B                           18.6358  1  1.582e-05 ***
C                            6.7366  2   0.034449 *
A:B                          4.5702  1   0.032532 *
A:C                          0.0142  2   0.992915
B:C                          0.6303  2   0.729685
A:B:C                        2.8599  2   0.239319


The main effects of A, B, C are sig, and so is the A:B two-way interaction.

However, when I calculate odds ratios and corresponding 95% confidence intervals for these effects, the CIs ALL include 1.

This is the code I use to get ORs and CIs:

#CI and OR for variable A
exp(summary(acc_model)$coefficients["A1",1] + qnorm(c(0.025,0.5,0.975)) * summary(acc_model)$coefficients["A1",2])


What can I do here? Is it okay to interpret the Anova() output? I suspect the significance of main effects A, B, and C in the Anova() output could be due to that being a Type II ANOVA (i.e., interactions including a given variable are not controlled for when the main effect of that variable is being evaluated, if I get that correclty), however, the CI for the OR of the A:B interaction ALSO contains 1, and that is NOT involved in any significant higher order interactions.

NOTE: when I use lmerTest, and get summary(acc_model), neither of these variables is significant. I would like to avoid using lmerTest, however, because that does not give an omnibus p-value for categorical variables with more than 2 levels (e.g., Variable C here, that would have 2 separate lines for the Reference_Category - Level1 and the Reference_Category - Level2 contrasts), so anova() and Anova() are neater.

If you use the confint method in R, there are three confint procedures for mixed models (merMod objects in the nlme package). Wald (Type III, wrong), profile, and bootstrap. Try profile and bootstrap and report back the findings. Note they still may not agree with Type II SS, but the reasons are more refined.
• @MGy how many $N$? did you try setting boot replicates very low? – AdamO Aug 26 at 21:18