It's well known ANOVA and linear regression are effectively the same analysis.

With that in mind, is there any point at all in using ANOVA to pick features to later fit a linear regression model?

My guess is no: It's like fitting linear regression, choosing the features that work well, and then fittting a linear regression model again only with those features.

I see students and candidates often doing this, and keep wondering why, and if there is any value in this approach at all.

  • 1
    $\begingroup$ As your link suggests, try replacing 'ANOVA' by 'linear regression' in your question (Anova is a red herring). $\endgroup$ – user603 Oct 26 '16 at 16:20
  • $\begingroup$ Yes @user603. That's what I was thinking, i.e. there is really no point at all in using linear regression to choose features and then using linear regression again with the hope that one gets a better fit? Is it fair to say that the second fit would most likely be worse (or no better) than the first one? $\endgroup$ – Amelio Vazquez-Reina Oct 26 '16 at 16:53

There's no point at all in doing that- as you mentioned they are both general linear models. However they can provide different insights, for example while linear regression focuses on the significant predictors, Anova output focuses on whether/where the group differences are (with performing adhoc analysis). linear Regression does not provide this.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.