I am running a generalized linear mixed-effects model in R using the glmer function of lme4. The outcome variable is trial-level accuracy in a task (incorrect trials are 0, correct trials are 1), and I have 3, trial-level predictors:
- A, a binary categorical variable
- B, another binary categorical variable
- C, a categorical variable with 3 levels
(All variables, including accuracy are factors.)
Here is the model I'm running:
acc_model <- glmer(accuracy ~ 1 + A*B*C + (1|subid), data = acc_data, family = binomial, control = glmerControl(optimizer ="bobyqa")) #subid simply refers to subject IDs as trials are nested within participants #I use bobyqa because the model failed to converge with the default optimizer
Then I use the Anova() function from the "car" package to see if the terms are significant.
This has worked nicely for me in the past (anova() does not give p-values for glmer models), and the results seem sensible in this case too:
Analysis of Deviance Table (Type II Wald chisquare tests) Response: acc Chisq Df Pr(>Chisq) A 10.0468 1 0.001526 ** B 18.6358 1 1.582e-05 *** C 6.7366 2 0.034449 * A:B 4.5702 1 0.032532 * A:C 0.0142 2 0.992915 B:C 0.6303 2 0.729685 A:B:C 2.8599 2 0.239319
The main effects of A, B, C are sig, and so is the A:B two-way interaction.
However, when I calculate odds ratios and corresponding 95% confidence intervals for these effects, the CIs ALL include 1.
This is the code I use to get ORs and CIs:
#CI and OR for variable A exp(summary(acc_model)$coefficients["A1",1] + qnorm(c(0.025,0.5,0.975)) * summary(acc_model)$coefficients["A1",2])
What can I do here? Is it okay to interpret the Anova() output? I suspect the significance of main effects A, B, and C in the Anova() output could be due to that being a Type II ANOVA (i.e., interactions including a given variable are not controlled for when the main effect of that variable is being evaluated, if I get that correclty), however, the CI for the OR of the A:B interaction ALSO contains 1, and that is NOT involved in any significant higher order interactions.
NOTE: when I use lmerTest, and get summary(acc_model), neither of these variables is significant. I would like to avoid using lmerTest, however, because that does not give an omnibus p-value for categorical variables with more than 2 levels (e.g., Variable C here, that would have 2 separate lines for the Reference_Category - Level1 and the Reference_Category - Level2 contrasts), so anova() and Anova() are neater.