Why is ANOVA taught / used as if it is a different research methodology compared to linear regression?

Question

ANOVA is equivalent to linear regression with the use of suitable dummy variables. The conclusions remain the same irrespective of whether you use ANOVA or linear regression.

In light of their equivalence, is there any reason why ANOVA is used instead of linear regression?

Note: I am particularly interested in hearing about technical reasons for the use of ANOVA instead of linear regression.

Edit

Here is one example using one-way ANOVA. Suppose, you want to know if the average height of male and females is the same. To test for your hypothesis you would collect data from a random sample of male and females (say 30 each) and perform the ANOVA analysis (i.e., sum of squares for gender and error) to decide whether an effect exists.

You could also use linear regression to test for this as follows:

Define: $\text{Gender} = 1$ if respondent is a male and $0$ otherwise. $$ \text{Height} = \text{Intercept} + \beta * \text{Gender} + \text{error} $$ where: $\text{error}\sim\mathcal N(0,\sigma^2)$

Then a test of whether $\beta = 0$ is a an equivalent test for your hypothesis.

Answer 1

As an economist, the analysis of variance (ANOVA) is taught and usually understood in relation to linear regression (e.g. in Arthur Goldberger's A Course in Econometrics). Economists/Econometricians typically view ANOVA as uninteresting and prefer to move straight to regression models. From the perspective of linear (or even generalised linear) models, ANOVA assigns coefficients into batches, with each batch corresponding to a "source of variation" in ANOVA terminology.

Generally you can replicate the inferences you would obtain from ANOVA using regression but not always OLS regression. Multilevel models are needed for analysing hierarchical data structures such as "split-plot designs," where between-group effects are compared to group-level errors, and within-group effects are compared to data-level errors. Gelman's paper [1] goes into great detail about this problem and effectively argues that ANOVA is an important statistical tool that should still be taught for it's own sake.

In particular Gelman argues that ANOVA is a way of understanding and structuring multilevel models. Therefore ANOVA is not an alternative to regression but as a tool for summarizing complex high-dimensional inferences and for exploratory data analysis.

Gelman is a well-respected statistician and some credence should be given to his view. However, almost all of the empirical work that I do would be equally well served by linear regression and so I firmly fall into the camp of viewing it as a little bit pointless. Some disciplines with complex study designs (e.g. psychology) may find ANOVA useful.

[1] Gelman, A. (2005). Analysis of variance: why it is more important than ever (with discussion). Annals of Statistics 33, 1–53.

Answer 2

I think Graham's second paragraph gets at the heart of the matter. I suspect it's not so much technical than historical, probably due to the influence of "Statistical Methods for Research Workers", and the ease of teaching/applying the tool for non-statisticans in experimental analysis involving discrete factors, rather than delving into model building and associated tools. In statistics, ANOVA is usually taught as a special case of regression. (I think this is similar to why biostatistics is filled with a myriad of eponymous "tests" rather than emphasizing model building.)

Answer 3

ANOVA can be used with categorical explanatory variables (factors) that take more than 2 values (levels), and gives a basic test that the mean response is the same for every value. This avoids the regression problem on carrying multiple pairwise t-tests between those levels:

• Multiple t-tests on a fixed 5% significance level, would make roughly 5% of them give wrong results.
• These tests are not independed from each other. Comparing A's levels with B's is connected with comparing A's to C's, as A's data are used in both tests.

It is better to use contrasts for different combinations on the factor levels you want to test.

Answer 4

I would say that some of you are using the term regression when you should be using general linear model. I think of regression as a glm that involves continuous covariates. When continuous covariates are combined with dummy variables that should be called analysis of covariance. If only dummy variables are used we refer to that special form of glm as analysis of variance. I think analysis of variance has a distinct second meaning as the procedure for testing for significant coefficients in a glm using the decomposition of variance into model term components and the error term component.