5
$\begingroup$

When analysing mixed-effects data using lmer() I find that using anova(type='marginal') and anova(type='III') give different results. Why the discrepancy?

The results from anova(type='marginal') are identical to those I get from using car:Anova(type='III') on the same model and to using both anova(type='marginal') and car:Anova(type='III') on the same data fitted using lme().

library(lme4)
library(nlme)
library(car)

# Some data
group<-as.factor(c(rep('A',32),rep('B',32)))
time<-as.factor(rep(c(rep('t1',16),rep('t2',16)),2))
cond<-as.factor(rep(c(rep('c1',8),rep('c2',8)),4))
subject<-as.factor(rep(c(rep(1:8,2),rep(9:16,2)),2))
set.seed(1)
dv<-c(rnorm(16, mean=3),rnorm(16, mean=1),rnorm(16, mean=0),rnorm(16, mean=3))
data<-data.frame(dv,group,time,cond, subject)

# Model using both lme() and lmer()
model_lme<-lme(dv~group*time*cond, random=~1|subject, data = data, method='ML')
model_lmer<-lmer(dv~group*time*cond+ (1|subject), data = data, REML=FALSE)

# Investigate output of same model with lme() og lmer()
'lme() anova(type=marginal):'
anova(model_lme, type='marginal')
'lme() car::Anova(type=III):'
Anova(model_lme,type='III')
'lmer() anova(type=III):'
anova(model_lmer, type='III')
'lmer() anova(type=marginal):'
anova(model_lmer, type='marginal')
'lmer() car::Anova(type=III):'
Anova(model_lmer,type='III')
$\endgroup$

1 Answer 1

6
$\begingroup$

This is admittedly confusing, but there are a bunch of differences/limitations between anova() and car::Anova() and between lme and lmer fits. tl;dr probably best to use car::Anova() for consistent results across model types.

  • anova on lme: allows type="sequential" or type="marginal" (only). type="marginal" should be closest to type-3. Returns F-statistics with denominator degrees of freedom (ddf) calculated by "inner-outer" method (group/parameter counting).
  • Anova on lme: allows type="II" or type="III". Returns chi-square statistics (i.e. infinite denominator df/no finite-size correction) only.
  • anova on lmer: returns sequential F statistics, with no p-values, unless you have the lmerTest package loaded, in which case you get a choice of type II vs III and a choice of ddf calculations
  • Anova on lmer: if you fit with REML=TRUE you can specify test="F" and get a choice of ddf calculations.

So reviewing the tests you did:

  • anova(model_lme, type='marginal'): type-III/marginal F tests, inner-outer ddf
  • Anova(model_lme,type='III'): similar, but returns chi-square statistics/p-values instead, so the p-values are slightly anti-conservative (e.g. p-value for time effect is 0.0012 for F(42) and $1.49 \times 10^{-5}$ for chi-square)
  • anova(model_lmer, type='III'): the type argument is ignored, so you get sequential/type-I F values (similar but ??? not identical to anova(model_lme, type='sequential')
  • anova(model_lmer, type='marginal'): ditto
  • Anova(model_lmer,type='III'): type-3, but chi-square: identical to Anova(model_lme, type='III')

If you refit both models with REML, Anova(model_lmer,type="III",test="F") and anova(model_lme,type="marginal") give similar results.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.