Common data-based variable selection procedures (for example, forward, backward, stepwise, all subsets) tend to yield models with undesirable properties, including:
- Coefficients biased away from zero.
- Standard errors that are too small and confidence intervals that are too narrow.
- Test statistics and p-values that do not have the advertised meaning.
- Estimates of model fit which are overly optimistic.
- Included terms which can be meaningless (e.g., exclusion of lower-order terms).
Yet, variable selection procedures persist. Given the problems with variable selection, why are these procedures necessary? What motivates their use?
Some proposals to start the discussion....
- The desire for interpretable regression coefficients? (Misguided in a model with many IVs?)
- Eliminate variance introduced by irrelevant variables?
- Eliminate unnecessary covariance/redundancies among the independent variables?
- Reduce the number of parameter estimates (issues of power, sample size)
Are there others? Are the problems addressed by variable selection techniques more or less important than the problems variable selection procedures introduce? When should they be used? When should they not be used?