0
$\begingroup$

let's suppose; I am fitting some models to a data set:

m1=lm(y~x1,data=data); m2=lm(y~x1+x2,data=data)
m3=lm(y~x1+x3, data=data); m4=lm(y~x2+x3, data=data)
m5=lm(y~x1+x2+x3, data=data)

I want to understand the use of anova() to compare different models.

My questions are:

  1. As discussed model compare by anova test, I need to put more general model later. So is anova(m2, m1) a wrong way of doing comparision?
  2. anova() can be used for nested models (models have a shared set of predictor variables and the same outcome variable, but one model has one or more additional predictor variables), can we use anova(m2, m3) or anova(m1, m4)?
  3. How to interpret output of anova() using three or more models? For example; anova(m1, m2, m3) or anova(m1, m2, m3, m4).

-

$\endgroup$
2
  • $\begingroup$ Update: in 2019, a new package 'performance' is released and as on date, we have version 0.7.2. This package have cool functions not only to compare performance of various models but also for checking heteroskedasticity, normality of residual, autocorrelation, colinearity, outliers, model specification etc. This model also provides other performance metrics like r-squared, AIC, BIC, RMSE, ICC or LOOIC. Basically, no need to use separate package for these. rdocumentation.org/packages/performance/versions/0.7.2 $\endgroup$ May 28 '21 at 2:35
  • $\begingroup$ Using performance package for model comparisons: github.com/arora123/R-for-Data-Science/blob/master/… $\endgroup$ May 28 '21 at 2:35
1
$\begingroup$
  1. If you do anova(m2, m1) it computes the same p-value as anova(m1, m2) so the order is not crucial. However, in the former case the change in degrees of freedom becomes negative which might look confusing in the output. (As pointed out in the other Q/A you referred to.)
  2. No. It is technically possible to enter anova(m2, m3) but the result is not a valid ANOVA. If you want to compare these models, you can either resort to information criteria (AIC, BIC, ...) or use dedicated tests for non-nested hypotheses (e.g., encompassing test, Vuong test, Cox test, J test, etc.).
  3. If you carry out anova(m1, m2, m5) this reports two tests which essentially correspond to anova(m1, m2) and anova(m2, m5). However, the error sum of squares is taken from m5 in both of these comparisons which might lead to small differences. Due to 2. the sequence of models must be nested in each step to return valid ANOVAs. But you can do an encompassing test between m2 and m3 by going through m5, i.e., anova(m2, m5, m3).
$\endgroup$
5
  • $\begingroup$ Thanks Achim. So, for anova(m1, m2, m5), we can infer that one of the models in increasing/decreasing order of RSS. E.g., The anova output on r-bloggers.com/r-tutorial-series-anova-tables can be interpreted as: Model 2 has lower RSS than Model 1 and Model 5 has Lowest RSS among the three models. Please correct, if I'm wrong. $\endgroup$ Jun 6 '16 at 11:53
  • $\begingroup$ Please also share some resource for learning non-nested hypotheses (e.g., encompassing test, Vuong test, Cox test, J test, etc.). Thanks again $\endgroup$ Jun 6 '16 at 11:54
  • $\begingroup$ Interpretation of ANOVA: For concluding that the RSS decreases you do not need a significance test. The point is whether it decreases significantly or not. In the blog post you link adding HGRAD in addition to UNEM improves the model significantly. And adding INC to the other two variables improves the model significantly again. As for non-nested hypotheses: This is covered, for example, in Econometric Analysis by Greene. $\endgroup$ Jun 6 '16 at 14:19
  • $\begingroup$ And since @learner apparently uses R s/he might consider looking at the lmtest package and its documentation. $\endgroup$
    – mdewey
    Jun 6 '16 at 14:25
  • $\begingroup$ Thanks.Yeah, I meant statistically significant decrease in RSS $\endgroup$ Jun 7 '16 at 5:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.