1
$\begingroup$

Suppose I have ten predictors that I want to use to predict Y. I plan to run a multiple regression using stepwise selection. My question is, should I assess colinearity in the final model (i.e., examine VIF of the variables that make it into the model)? Or should I first simply run a multiple regression with all ten variables and examine colinearity when all ten are in the model?

$\endgroup$
7
  • 2
    $\begingroup$ The first question is why you plan to use stepwise selection in the first place. This superb answer, among many others on this site, explains the substantial drawbacks. Please explain more about what you are trying to accomplish in your work; there is almost certainly a better solution than stepwise selection. $\endgroup$
    – EdM
    Commented Nov 30, 2016 at 20:45
  • $\begingroup$ Okay, I am looking into Least Angle Regression as a model selection procedure. But this is a separate issue. I'm still hoping to get an answer to my original question: should I evaluate colinearity among all possible predictors or just those that make it into the model? $\endgroup$
    – Dave
    Commented Nov 30, 2016 at 22:20
  • 1
    $\begingroup$ Possible duplicate of Algorithms for automatic model selection $\endgroup$
    – mkt
    Commented Sep 12, 2019 at 17:46
  • 1
    $\begingroup$ @mkt But this is an interesting question and in his comment, the OP says he is looking into LAR. So ... what to do? Maybe a new question is needed. $\endgroup$
    – Peter Flom
    Commented Sep 14, 2019 at 14:35
  • 1
    $\begingroup$ @PeterFlom I'm fine with leaving open since your answer covers the stepwise problem and also addresses the interesting portion (+1). Would perhaps be nice for the LAR part to be edited into the question, but I'm not sure if that is best left to the OP. $\endgroup$
    – mkt
    Commented Sep 14, 2019 at 14:42

1 Answer 1

1
$\begingroup$

Per the comments - if you plan to run stepwise regression, you should probably change your plans.

But, suppose (as you indicate in a comment) that you are going to use LAR. Then the question is whether looking at collinearity first is a good idea. I would say yes. Collinearity is going to affect the standard errors of your parameter estimates and that will affect all sorts of things. If there is substantial collinearity, then you might want to apply your variable selection method to a different model - such as ridge regression. Or you might want to remove one or more variables from the data set. Or something else.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.