51
$\begingroup$

From the documentation for anova():

When given a sequence of objects, ‘anova’ tests the models against one another in the order specified...

What does it mean to test the models against one another? And why does the order matter?

Here is an example from the GenABEL tutorial:

    >  modelGen = lm(qt~snp1)
    >  modelAdd = lm(qt~as.numeric(snp1))
    >  modelDom = lm(qt~I(as.numeric(snp1)>=2))
    >  modelRec = lm(qt~I(as.numeric(snp1)>=3))
     anova(modelAdd, modelGen, test="Chisq")
    Analysis of Variance Table

    Model 1: qt ~ as.numeric(snp1)
    Model 2: qt ~ snp1
      Res.Df  RSS Df Sum of Sq Pr(>Chi)
    1   2372 2320                      
    2   2371 2320  1    0.0489     0.82
     anova(modelDom, modelGen, test="Chisq")
    Analysis of Variance Table

    Model 1: qt ~ I(as.numeric(snp1) >= 2)
    Model 2: qt ~ snp1
      Res.Df  RSS Df Sum of Sq Pr(>Chi)
    1   2372 2322                      
    2   2371 2320  1      1.77     0.18
     anova(modelRec, modelGen, test="Chisq")
    Analysis of Variance Table

    Model 1: qt ~ I(as.numeric(snp1) >= 3)
    Model 2: qt ~ snp1
      Res.Df  RSS Df Sum of Sq Pr(>Chi)  
    1   2372 2324                        
    2   2371 2320  1      3.53    0.057 .
    ---
    Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 

How do I interpret this output?

$\endgroup$

2 Answers 2

48
$\begingroup$

When you use anova(lm.1,lm.2,test="Chisq"), it performs the Chi-square test to compare lm.1 and lm.2 (i.e. it tests whether reduction in the residual sum of squares are statistically significant or not). Note that this makes sense only if lm.1 and lm.2 are nested models.

For example, in the 1st anova that you used, the p-value of the test is 0.82. It means that the fitted model "modelAdd" is not significantly different from modelGen at the level of $\alpha=0.05$. However, using the p-value in the 3rd anova, the model "modelRec" is significantly different form model "modelGen" at $\alpha=0.1$.

Check out ANOVA for Linear Model Fits as well.

$\endgroup$
4
  • 1
    $\begingroup$ But does that imply anything about whether one of the two is better? Thanks! $\endgroup$
    – qed
    Mar 29, 2013 at 17:22
  • 5
    $\begingroup$ It depends on how you define the term "better". If you define it as the model that provides less residual sum of squares, then the answer is yes. This is because, this test compares the reduction in the residual sum of squares. $\endgroup$
    – Stat
    Mar 29, 2013 at 21:32
  • 4
    $\begingroup$ On the other hand, if the two models are not significantly different, could one argue that the simpler model is "better"? I am thinking about parcimony here. $\endgroup$
    – Sininho
    May 31, 2016 at 15:00
  • 1
    $\begingroup$ what if the anova(mod1, mod2, test = "LRT") what's the difference does this make? $\endgroup$
    – ElleryL
    Aug 13, 2017 at 21:36
3
$\begingroup$

I agree with the OP that the help for the anova() function is not particularly helpful. The help of anova.lm() (as suggested by @Stat) is much more informative.

For me, however, the real eye-opener was "YaRrr! The Pirate's Guide to R", Chap.15.3:

The anova() function will take the model objects as arguments, and return an ANOVA testing whether the more complex model is significantly better at capturing the data than the simpler model. If the resulting p-value is sufficiently low (usually less than 0.05), we conclude that the more complex model is significantly better than the simpler model, and thus favor the more complex model. If the p-value is not sufficiently low (usually greater than 0.05), we should favor the simpler model.

This explains pretty well (IMO) the strategy behind the comparison, taking model complexity into account.

Note that there's another use of the anova() function, namely to compare non-linear regression fits, as explained in this excellent CV post. Briefly, you compare a "group-wise" nonlinear regression fit to a "pooled" fit (same data, same functional form of course). If the anova() function gives you a "significant difference", then you conclude that the groupwise fit describes your data better than the pooled fit. If there's "no significant difference", then the groupwise fits are equivalent to the pooled fit, i.e. "no difference between the curves".

$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.