3
$\begingroup$

What is the advantage (if any) to fitting both a linear regression model to a dataset and running an ANOVA test if these are both ultimately identical?

$\endgroup$
3
  • 3
    $\begingroup$ The premise of this question is that it is good to do so, but what makes you think so? There is no rule that says you should do both. However, it might be used in a class to demonstrate that ANOVA is a linear model. $\endgroup$ Nov 8, 2017 at 5:59
  • $\begingroup$ The question entailed no such premise. I am just trying to understand whether there might be a reason to report one but not the other, and why people sometimes do. $\endgroup$
    – Namenlos
    Nov 8, 2017 at 6:02
  • 1
    $\begingroup$ Linear regression is ANOVA. $\endgroup$
    – Peter Flom
    Nov 8, 2017 at 20:50

1 Answer 1

6
$\begingroup$

Tested hypotheses in ANOVA and linear regression are different.

  • Linear model gives one estimates of the coefficient values along with the significance of their being different from zero. In this way one can understand how each of the input variables (as factor levels if we refer to a linear model with categorical input variable) influences the mean value of a continuous dependent variable.

So the null hypothesis is being formulated for each input variable as to whether they differ from zero. Additionally you can formulate a similar one-sided hypothesis for R Squared value.

  • ANOVA's null hypothesis is whether group mean values of the dependent variable are not significantly different, while an alternative hypothesis is just that at least one of the factor level forms a group of observations which mean value is different from overall mean. ANOVA does not specify which of the factor levels are significant discriminators.

If ANOVA test results in rejecting the null hypothesis, you can build a significant linear regression model, but before you build one, you don't understand which coefficient(s) will be significantly different from zero.

Besides, Fisher's (F) statistic used in ANOVA applies to measuring a significant decrease in Residual Sum of Squares attributed to the model one has built in comparison with intercept-only model.

$\endgroup$
1
  • $\begingroup$ I am quite impressed by this concise comment. Thumbs up :) $\endgroup$
    – Kevin Kang
    May 10, 2018 at 9:16

Not the answer you're looking for? Browse other questions tagged or ask your own question.