Why do anova(type='marginal') and anova(type='III') yield different results on lmer() models?

Question

When analysing mixed-effects data using lmer() I find that using anova(type='marginal') and anova(type='III') give different results. Why the discrepancy?

The results from anova(type='marginal') are identical to those I get from using car:Anova(type='III') on the same model and to using both anova(type='marginal') and car:Anova(type='III') on the same data fitted using lme().

library(lme4)
library(nlme)
library(car)

# Some data
group<-as.factor(c(rep('A',32),rep('B',32)))
time<-as.factor(rep(c(rep('t1',16),rep('t2',16)),2))
cond<-as.factor(rep(c(rep('c1',8),rep('c2',8)),4))
subject<-as.factor(rep(c(rep(1:8,2),rep(9:16,2)),2))
set.seed(1)
dv<-c(rnorm(16, mean=3),rnorm(16, mean=1),rnorm(16, mean=0),rnorm(16, mean=3))
data<-data.frame(dv,group,time,cond, subject)

# Model using both lme() and lmer()
model_lme<-lme(dv~group*time*cond, random=~1|subject, data = data, method='ML')
model_lmer<-lmer(dv~group*time*cond+ (1|subject), data = data, REML=FALSE)

# Investigate output of same model with lme() og lmer()
'lme() anova(type=marginal):'
anova(model_lme, type='marginal')
'lme() car::Anova(type=III):'
Anova(model_lme,type='III')
'lmer() anova(type=III):'
anova(model_lmer, type='III')
'lmer() anova(type=marginal):'
anova(model_lmer, type='marginal')
'lmer() car::Anova(type=III):'
Anova(model_lmer,type='III')

Answer 1

This is admittedly confusing, but there are a bunch of differences/limitations between anova() and car::Anova() and between lme and lmer fits. tl;dr probably best to use car::Anova() for consistent results across model types.

• anova on lme: allows type="sequential" or type="marginal" (only). type="marginal" should be closest to type-3. Returns F-statistics with denominator degrees of freedom (ddf) calculated by "inner-outer" method (group/parameter counting).
• Anova on lme: allows type="II" or type="III". Returns chi-square statistics (i.e. infinite denominator df/no finite-size correction) only.
• anova on lmer: returns sequential F statistics, with no p-values, unless you have the lmerTest package loaded, in which case you get a choice of type II vs III and a choice of ddf calculations
• Anova on lmer: if you fit with REML=TRUE you can specify test="F" and get a choice of ddf calculations.

So reviewing the tests you did:

• anova(model_lme, type='marginal'): type-III/marginal F tests, inner-outer ddf
• Anova(model_lme,type='III'): similar, but returns chi-square statistics/p-values instead, so the p-values are slightly anti-conservative (e.g. p-value for time effect is 0.0012 for F(42) and $1.49 \times 10^{-5}$ for chi-square)
• anova(model_lmer, type='III'): the type argument is ignored, so you get sequential/type-I F values (similar but ??? not identical to anova(model_lme, type='sequential')
• anova(model_lmer, type='marginal'): ditto
• Anova(model_lmer,type='III'): type-3, but chi-square: identical to Anova(model_lme, type='III')

If you refit both models with REML, Anova(model_lmer,type="III",test="F") and anova(model_lme,type="marginal") give similar results.