8
$\begingroup$

On the Mathematics site an OP who is just learning statistics gave his description of the difference between linear regression and ANOVA and asked if his interpretation was correct. I responded that linear regression considers how a set of covariates relate to a response in a functional form (could have added "that is linear in the parameters") whereas ANOVA categorizes the response into a class or classes of group(s) and tests for a difference between group means. A member downvoted my answer saying that ANOVA can include continuous predictors as well. His own answer indicated that he was considering the term ANOVA to mean the testing of significant terms from the decomposition of variance in the general linear model. We both gave descriptions of linear regression that agreed.

My question is: "What do you think is the best answer? His answer? My answer? An explanation providing the two meanings of ANOVA? Something else?

Link: https://math.stackexchange.com/questions/183704

Improve this question
$\endgroup$
16
  • 1
    $\begingroup$ Can you provide the link to the discussion? Also we have an anova tag, but I don't have sufficient rep to suggest/create a synonym with your created analysis-of-variance tag, could you create the synonym? $\endgroup$
    – Andy W
    Aug 18, 2012 at 13:28
  • 3
    $\begingroup$ Also there is a related discussion in a prior question on the site (although not a duplicate), Why is ANOVA taught / used as if it is a different research methodology compared to linear regression? $\endgroup$
    – Andy W
    Aug 18, 2012 at 13:29
  • 1
    $\begingroup$ @smillig The discussion was not about computer routines. We just discussed SAS in my case and R in his as support to our positions. It is common to refer to the variance decomposition table in the general linear model as the ANOVA table. But ANOVA is usually differentiated from regression in the way I stated. Thanks for linking the other post. $\endgroup$
    – Michael R. Chernick
    Aug 18, 2012 at 13:59
  • 7
    $\begingroup$ I can't understand why nobody uses the purpose argument, i.e. that linear regression is a modelling method and ANOVA is a hypothesis testing framework. I know that they can be extended in a way they overlap, but the canonical formulations are perfectly separated. $\endgroup$
    – user88
    Aug 18, 2012 at 14:26
  • 2
    $\begingroup$ In ANOVA, we are trying to find how much of the variance is accounted for our manipulation of the independent variables. In multiple regression, we do not directly manipulate the independent variables, but instead just measure the naturally levels of variables and see if this helps us to predict the dependent variable. However, ANOVA and multiple regression are fundamentally the same since both of them try to explain the variance in the level of one variable on the basis of the level of one or more other variables. Ref: "SPSS for Psychologists: A Guide to Data Analysis Using Spss for Windows" $\endgroup$
    – Stat
    Aug 19, 2012 at 4:42

1 Answer 1

Reset to default
2
$\begingroup$

ANOVA and Linear regression are twin princesses grown in different castles. Please see the book of Andy Field: Discovering statistics using SPSS. He has a very nice explanation including the evolution in time of this two. Anyway put bluntly: they are very similar and developed in parallel for a certain period of time by different scientific communities. NB: Of course the comparison is software independent.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Your reference to Andy Field's book is extremely helpful, thank you! The discussion you are referring to is at page ~350 in the 3rd edition. $\endgroup$
    – cmo
    Oct 2, 2014 at 19:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.