So I regularly make use of lme4 and lmerTest to fit Linear Mixed Models. I then do the post ANOVA contrasts via the emmeans (Estimated Marginal Means) package.

However, I understand that for Generalized Linear Mixed Models, the ANOVA (or analysis of deviance) table does not provide any p-values at all. Is it still valid to perform the contrasts without this?

One method I read about is Cluster Bootstrapping the GLMM. I found a nice package called ClusterBoot which seems to perform the cluster bootstrap. It was released recently: https://link.springer.com/article/10.3758/s13428-019-01252-y

I am able to get bootstrap coefficients, SEs, and percentile/parametric intervals for all the coefficients with this. However, this is difficult to interpret since I don't know how to use all of this to get ANOVA contrasts for my pairwise comparisons from this. Emmeans interfaces nicely with lme4 and provides this but can't be used with this.

How would I go about getting contrasts knowing just the Bootstrap Coefficients/SEs? Along with corresponding p-values.

  • $\begingroup$ I don’t see any particular issue with using emmeans directly from your glmer fitted model, as long as the model is appropriate and fits well. It matters not whether the model summary or anova table has p values. $\endgroup$ – rvl Aug 27 '19 at 23:43
  • $\begingroup$ @rvl Thanks! For some reason, in intro stat classes they emphasized doing ANOVA before post-hoc contrasts and so I've always done things this way and this was never cleared up in future classes. In that case, can I potentially bootstrap the emmeans results using bootMer along with a function that outputs the Mean values for the pairwise differences estimated by emmeans? I basically made the $contrasts into a dataframe and took mean values and bootstrapped these. (This is partially also for learning purposes, so wondering if this is a valid approach) $\endgroup$ – Vattaka Aug 28 '19 at 6:52
  • $\begingroup$ I don’t see too much point in doing a bootstrap that way, but I suppose there is no harm in it. As I understand, you are not sampling from the observed data, just from the reported sampling distribution. So it will just replicate the results you see. $\endgroup$ – rvl Aug 28 '19 at 10:30
  • $\begingroup$ @rvl Oh I see then its not doing what I expected it to be doing which is resample from the data. I made my own code to resample the cluster (in this case Mice) and fitted the lmer. It kept complaining about rank deficiencies and dropping columns which is expected I think for resampling. But then suddenly the emmeans contrast function said "Nonconforming number of contrast coefficients" but works on the full dataset fine and earlier runs. The emmeans also works fine. Is there a reason why I'd get this error for contrast resampling? Basically this stopped the loop. Thanks for the help. $\endgroup$ – Vattaka Aug 28 '19 at 18:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.