6
$\begingroup$

So just correct me where I go wrong, but my understanding of the differences between ANOVA and regression analysis is that ANOVA is comparing two independent groups while regression is used on one group with independent observations.

But then my question is:

Imagine we have two groups of patients, group A and group B. We have variable "x" that we want to investigate whether there exists a difference in between these two groups.

Now we could do an ANOVA but what would be the difference between doing this and simply making a binary variable with 0 denoting group A and 1 denoting group B and then running a regression with this variable as the dependent and "x" as the independent?

Thank you!

$\endgroup$
6
  • 1
    $\begingroup$ Run a simulation in R and see what happens! Do you get the same p-value from the ANOVA as you get from the parameter on “x” in your logistic regression? $\endgroup$
    – Dave
    Commented Nov 25, 2019 at 10:50
  • $\begingroup$ I'm using STATA and I ran: "Logistic y x" and got p = 0.482 I then ran: "Anova y x" and got p = 0.543 y is a binary variable and x is a continuous variable. $\endgroup$
    – Paze
    Commented Nov 25, 2019 at 10:54
  • $\begingroup$ Did you give two distinct distributions? Those p-values are large, even for the standard ANOVA technique. $\endgroup$
    – Dave
    Commented Nov 25, 2019 at 10:58
  • $\begingroup$ It's the exact same variables in both cases. $\endgroup$
    – Paze
    Commented Nov 25, 2019 at 11:43
  • $\begingroup$ For each method, are you using separable groups? You are not able to reject a null hypothesis of equality with p-values around 1/2. $\endgroup$
    – Dave
    Commented Nov 25, 2019 at 11:53

1 Answer 1

5
$\begingroup$

ANOVA and logistic regression have different aims. A bit loosely speaking, ANOVA uses a continuous response variable and predicts the value of that variable, while logistic regression uses a binary response variable and predicts the category. ANOVA then attempts to find the mean of the response variable, conditioned on the group membership.

You can use a classifier such as logistic regression to see how well you can separate your data. In that case, you would invert the problem: instead of predicting the continuous measurements from the group membership, predict the group membership from the continuous measurements. To get the p-value, look at the parameter on the continuous variable. While I am unfamiliar with Stata, R uses Wald confidence intervals by default, and I have been doing logistic regression parameter inference lately via likelihood ratio testing. A parameter significantly different from zero indicates that the variable gives insight into the group to which the observation belongs.

By the way, ANOVA is a linear regression. We have some posts about this fact. I will link one that I liked.

https://stats.stackexchange.com/a/76292/247274

EDIT I found another: https://stats.stackexchange.com/a/16956/247274. I will note that, when there are only two groups, this is identical to the equal-variance t-test. (Remember that R uses the unequal-variance Welch test by default, and other software may, too.) Allowing for some numerical imprecision on a computer, the F-stat from the method in this link and the square of the t-test's t-stat will be the same, and the resulting p-values will be the same.

$\endgroup$
5
  • 1
    $\begingroup$ "By the way, ANOVA is a linear regression. We have some posts about this fact. I will link one that I liked." I guess what I'm asking why ANOVA exists if linear regression also exists? $\endgroup$
    – Paze
    Commented Nov 25, 2019 at 21:25
  • 2
    $\begingroup$ Fair question. I think it's because it's conceptually easier to extend the t-test of means to multiple groups than it is to figure out what an F-test really does and how it reduces variability once we condition a multivariate distribution on some covariates (the predictors). For better or for worse, most users of statistical methods just want to get an answer that they can put in their presentations to their bosses. $\endgroup$
    – Dave
    Commented Nov 25, 2019 at 21:30
  • $\begingroup$ @Paze Or you could think of the classic ANOVA k-sample test as a special case of linear regression. Likewise, we can see the two-sample t-test as a special case of ANOVA and, therefore, of linear regression. $\endgroup$
    – Dave
    Commented Nov 25, 2019 at 21:34
  • $\begingroup$ Thank you, is there a simple way to know when to use one over the other? $\endgroup$
    – Paze
    Commented Nov 26, 2019 at 11:01
  • $\begingroup$ They're the same technique, so stick to the terminology conventions of your field. If all you have are the groups, then the term is very likely to be ANOVA (or t-test in the two-sample case). $\endgroup$
    – Dave
    Commented Nov 26, 2019 at 14:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.