# ANOVA model comparison in R: AIC, BIC, LogLik vs RSS

When comparing 2 linear models in R with anova(mod1, mod2), I used to get a nice output showing AIC, BIC, LogLik etc:

  Df AIC BIC logLik deviance Chisq Chi Df Pr(>Chisq)
mod2 11 847 877   -412      825
mod1 12 849 882   -412      825     0      1          1


However, I recently got a new computer, and with a fresh R install I now get a different output from running the same R markdown:

  Res.Df    RSS Df Sum of Sq      F  Pr(>F)
1     26 72.304
2     25 42.213  1    30.091 17.821 0.00028 ***


Don't worry about the specific values, these are from different datasets, but my problem is that I can't see how to change the format of the anova output so that I get the AIC, BIC etc. There are plenty of examples on the web of model comparison anova's, and it seems an even split between the type of output (RSS v AIC etc). But no-one addresses the question of how to change output preferences. In fact, I have no idea how I happened to get the original output in the first place. Any suggestions? I presume that there is some package (not) running to change the output of anova(mod1, mod2).

I suspect you are running a linear mixed model and not a linear model, in both R 4.0 and 3.6.2, i get the same output:

mod2 = lm(mpg ~ hp + carb + cyl , data=mtcars)
mod1 = lm(mpg ~ hp +carb , data=mtcars)
anova(mod1,mod2)

Analysis of Variance Table

Model 1: mpg ~ hp + carb
Model 2: mpg ~ hp + carb + cyl
Res.Df    RSS Df Sum of Sq      F    Pr(>F)
1     29 445.19
2     28 288.74  1    156.45 15.171 0.0005564 ***


It is not normal for anova to produce the deviance estimate, and you honestly don't see it in the source code. If you run a linear mixed model using lme4 , you get the exact output as you showed:

library(lme4)
fm1 <- lmer(Reaction ~ Days + (Days | Subject), sleepstudy,REML=FALSE)
fm2 <- lmer(Reaction ~ 1 + (Days | Subject), sleepstudy,REML=FALSE)
anova(fm1,fm2)

Data: sleepstudy
Models:
fm2: Reaction ~ 1 + (Days | Subject)
fm1: Reaction ~ Days + (Days | Subject)
Df    AIC    BIC  logLik deviance  Chisq Chi Df Pr(>Chisq)
fm2  5 1785.5 1801.4 -887.74   1775.5
fm1  6 1763.9 1783.1 -875.97   1751.9 23.537      1  1.226e-06 ***


And if you apply the underlying anova function to your linear models now, you get the same:

lme4:::anova.merMod(mod2,mod1)
Data: mtcars
Models:
mod1: mpg ~ hp + carb
mod2: mpg ~ hp + carb + cyl
Df    AIC    BIC  logLik deviance  Chisq Chi Df Pr(>Chisq)
mod1  4 183.06 188.92 -87.530   175.06
mod2  5 171.21 178.53 -80.603   161.21 13.855      1  0.0001975 ***


I would check again whether you are running a linear model or mixed model. lme4:::anova.merMod is meant for mixed models, not fixed effect model that are created using lm or glm in R.

If it is indeed a fixed effect model, you can get the BIC and AIC output simply by doing:

res = data.frame(
do.call(merge,list(BIC(mod1,mod2),AIC(mod1,mod2))),
logLik=sapply(list(mod1,mod2),logLik),
anova(mod1,mod2,test='Chisq'))

df      BIC      AIC    logLik Res.Df      RSS Df Sum.of.Sq     Pr..Chi.
1  4 188.9238 183.0608 -87.53041     29 445.1936 NA        NA           NA
2  5 178.5346 171.2059 -80.60295     28 288.7448  1  156.4488 9.819659e-05


Using R version 4.0.2, the documentation says that you can indicate what test you wish to use.

anova(lm1, lm2) is the same as anova(lm1, lm2, test = "F") Other options are “Rao”, “LRT”, “Chisq”, “Cp”, NULL.