Why automated variable selection methods like stepwise regression, backward elimination and forward stepwise regression are not suitable when the regressors suffer from multicollinearity?
Could anyone give me some formal explanation of this fact?
Could these methods be used also for selecting the regressors in a logistic regression?
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
Stepwise, backward and forward aren't recommended whether the regressors are collinear or not. All the output will be wrong: p values will be too low, standard errors too small, parameter estimates biased away from 0 and more. See Frank Harrell's book Regression Modeling Strategies for extensive documentation of this.
If you have collinearity then you have additional problems. The parameter estimates will have high variance and be unstable.
Ways of dealing with this vary. One method is ridge regression, another is to delete some variables, another is to combine variables in some way.
answered Oct 17, 2015 at 11:54