3
$\begingroup$

My data has a binary response (correct/incorrect), one continuous predictor score, three categorical predictors (race, sex, emotion) and a random intercept for the random factor subj. All predictors are within-subject. One of the categorical factor has 3 levels, the other have two.

I need advice on obtaining "global" p-values for each categorical factor (in an "ANOVA like" way)

Here is how I proceed :

I fitted a binomial GLMM using 'glmer' from the lme4 package (because 'glmmML' doesn't compute on my data and glmmPQL does not provide AIC) and did model selection using drop1 repeatedly until no more terms can be dropped. Here is the final model (let's assume it has been validated):

library(lme4)
M5 <- glmer(acc ~ race + sex + emotion + sex:emotion + race:emotion + score +(1|subj), 
        family=binomial, data=subset)
# apparently using family with lmer is deprecated 
drop1(M5, test="Chisq")
summary(M5)

drop1 gives p-values for the higher level terms only (the two 2-way interactions + score). summarygives p-values for every term, but separates the different levels of each categorical factor.

How can I get "global" p-values for each factor? I need to report them even if they are not the most relevant or meaningful estimates of signifiance here. How should I proceed? I tried searching on the web and ended up reading about likelihood ratios or the "Wald test" but I am not sure if or how this would apply here.

(PS: This is a duplicate from my "anonymous" post here that needed editing: https://stats.stackexchange.com/questions/90487/binomial-mixed-model-with-categorical-predictors-model-selection-and-getting-p Sorry about that.)

Improve this question
$\endgroup$
  • 2
    $\begingroup$ use afex::mixed as in mixed(acc ~ race + sex + emotion + sex:emotion + race:emotion + score +(1|subj), family=binomial, data=subset, method = "LRT"). To obtain p-values based on parametric bootstrap, you can use method = "PB" (but you will need to set the number of samples, see help). Also, you most likely need random slopes for your within-subject factors. Your random effects structure seems unreasonable! $\endgroup$ – Henrik Mar 19 '14 at 0:46
  • 2
    $\begingroup$ did you look at ?pvalues in the lme4 package ... ? in addition to afex::mixed, it also suggests car::Anova and lmerTest::anova ... $\endgroup$ – Ben Bolker Mar 19 '14 at 2:42
  • $\begingroup$ @Ben Bolker : Thanks, I did not know about the ?pvalueshelp section. However, apparently neither car::Anova (which I had tried prior to posting) nor lmer::anovawork for GLMMs. $\endgroup$ – user42174 Mar 19 '14 at 8:23
  • $\begingroup$ @Henrik: many thanks, I'm trying that now. I favored the most simple random effect structure because I am unsure about how to specify the random slope properly (especially, should I update it as some interactions are dropped in the fixed effect part?). $\endgroup$ – user42174 Mar 19 '14 at 8:24
  • $\begingroup$ @Henrik : I tried mixed(acc ~ race + sex + emotion + sex:emotion + race:emotion + score +(1+sex|subj), family=binomial, data=subset, type = 3, method = "LRT") as suggested (after selecting random then fixed effects) but this gives an error: Fitting 8 (g)lmer() models: [. Erreur dans lme4::glFormula(formula = acc ~ 0 + m.matrix[, -1L] + (1 + sex | : rank of X = 10 < ncol(X) = 16 I don't think my design matrix is wrong. Have you encountered this error before? $\endgroup$ – user42174 Mar 19 '14 at 10:33
3
$\begingroup$

Copy-Paste of the answer from @Henrik:

use afex::mixed as in mixed(acc ~ race + sex + emotion + sex:emotion + race:emotion + score +(1|subj), family=binomial, data=subset, method = "LRT"). To obtain p-values based on parametric bootstrap, you can use method = "PB" (but you will need to set the number of samples, see help).

Also, you most likely need random slopes for your within-subject factors. Your random effects structure seems unreasonable!

Improve this answer
$\endgroup$
1
$\begingroup$

Use the library car

M5 <- glmer(acc ~ race + sex + emotion + sex:emotion + race:emotion + score +(1|subj), family=binomial, data=subset)

library(car)

Anova(Ms)
Improve this answer
$\endgroup$
  • 2
    $\begingroup$ OP says in comments that car::Anova doesn't work with glmer results. Did you test this? (OP may be mistaken, but I think it's worth saying that explicitly in your answer if they are.) $\endgroup$ – Ben Bolker Jul 9 '17 at 19:37
  • $\begingroup$ This is a bit late, but for posterity: yes, at the moment, car::Anova will produce p-values for glmer results. Note that the original question is now several years old, but car still (2017) does not appear to officially support glmer in its documentation. I'm not sure of the exact method it uses to obtain these p-values. $\endgroup$ – Stephen Oct 23 '17 at 16:59
  • 1
    $\begingroup$ Yes, I have done some analysis with repeated measurements design, with binomial answers, the Anova function of the car library carries out the analysis of Wald with the chi-square distribution. cran.r-project.org/web/packages/car/car.pdf $\endgroup$ – user168279 Nov 4 '17 at 5:10
-1
$\begingroup$ 
library(lme4)            
model=glmer(hospitalisation~age_cat+sexe+mectizan+doublelecture2+(1 | village), Loaloa, family = "binomial")
library(afex)  ###Global p-value
mixed(hospitalisation~age_cat+sexe+mectizan+doublelecture2+(1 | village), Loaloa, family = "binomial", method = "LRT")

library(broom.mixed) ###Other p-values (not global)
tidy(model,conf.int=TRUE,exponentiate=TRUE,effects="ran_vals")
Improve this answer
$\endgroup$
  • $\begingroup$ Hi @user289172, can you help add an explanation to the answer. I think it helps OP much better than just code $\endgroup$ – StupidWolf Jun 21 '20 at 20:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.