Linear Regression and ANOVA

Question

I found two very useful posts about the difference between linear regression analysis and ANOVA and how to visualise them:

Why is ANOVA taught / used as if it is a different research methodology compared to linear regression?

How to visualize what ANOVA does?

As stated in the first post, to test whether the average height of male and females is the same you can use a regression model ($y = \alpha + \beta x + \epsilon$, where $y$ denotes height and $x$ denotes gender) and test whether $\beta = 0$. If $\beta = 0$, then there is no difference in the height between males and females. However, I am not quite sure how this is tested when you have three groups. Imagine the following example:

height (y) -  group (x)
5          -  A
6          -  A
7          -  A
6          -  A
30         -  B
32         -  B
34         -  B
33         -  B 
20         -  C
19         -  C
21         -  C
22         -  C

The regression model would look like:

$$y = a+ b x + \epsilon$$

I quickly visualized the data (see image below)

They way I understood the regression model is that it would now test whether any of the three slopes (AB, AC or BC) has a slope $b$ which is significantly different from 0. If that's the case one can conclude like in an ANOVA that there is at least one group in which height is significantly different from one or more groups. Afterwards, one could use a post-hoc test of course to test which of the groups really differ. Is my understanding of how the regression models tests this hypothesis correct?

Answer 1

For categorical $X$ it is better to think of contrasts and group differences. Visualization per se doesn't help get this idea across, I think, other than to nicely show confidence limits in addition to point estimates of differences in means.

I do find that I almost never use ANOVA, in favor of linear models, except when I want to partition the sum of squares due to regression to describe, for example, how much of the variation in $Y$ is due to nonlinear effects in $X$ or is due to a subset of $X$s.

Regression models allow for covariate adjustment, and generalize ANOVA in a variety of ways. This is especially true for semiparametric models (e.g., proportional odds ordinal logistic model; Cox proportional hazards model) that generalize Wilcoxon/Kruskal-Wallis ideas and are much more robust than linear models.

Answer 2

Your basic question is not clear. However, it may be pointed out that regression does not provide us with a test statistic. It produces the effect-size for an independent variable(s). The ANOVA generates the test static implying that how much sampling error is there. This sampling error is compared with the expected sampling error. If the observed sampling error is converted into the form of t-statistic (or F-statistic) with a view to compare it with F-statistics if we have two factors (categorical variables). I doubt that given my level of readings and skills in statistics, I can give technical description.

Answer 3

If I understood you correctly, since you have three categories in your data, you need to put two dummy variables in your regression model. If beta1 and beta2 are not significants then you can conclude there is no difference in the means between the two groups. How about the third group?

It is easier to conduct ANOVA and then post hoc multiple comparison.