1
$\begingroup$

I have seen the following commonly used: 1. fit a model with all variables, 2. in a single reduction step, remove from the model all variables at once that do not fit some criteria (p-value, whatever), 3. calibrate the reduced model, in my case to a new data set, 4. check model results and hopefully everything went well and you can stop.

I'm betting this will perform better than stepwise, especially when used with data splitting, but would appreciate a name for this procedure if it exists and perhaps a reference related to it so that I can learn more. Maybe it is so simple/bad/obvious that no one has bothered? -WVG

Improve this question
$\endgroup$
5
  • $\begingroup$ Two correlated predictors will tend to have high p-values (neither explains much variation in the response given the presence of the other in the model), so you could end up removing both when stepwise would have kept one in, thus degrading predictive performance severely. $\endgroup$
    – Scortchi ♦
    Commented Oct 25, 2013 at 12:28
  • $\begingroup$ Thanks for the input. I have examined multicollinearity among predictors and it is very low for all variables. In my case, only 2 independent variables were eliminated, and they were not expected to be correlated (very different). I would rather eliminate them together in one step, but I would like to know a bit more. $\endgroup$
    – wvguy8258
    Commented Oct 25, 2013 at 12:32
  • 1
    $\begingroup$ If all predictors were orthogonal then the only effect of removing some would be to bias the estimate of the error variance, making the calculated p-values & confidence intervals misleading (as with stepwise); which hardly seems an advantage. There's plenty under the model selection tag (e.g. this & this) about the perils of model reduction & suggestions for methods that aren't terrible. $\endgroup$
    – Scortchi ♦
    Commented Oct 25, 2013 at 12:50
  • $\begingroup$ Does anyone know the name of this "all at once backward elimination" and whether it has been described/studied? $\endgroup$
    – wvguy8258
    Commented Oct 25, 2013 at 17:51
  • 1
    $\begingroup$ I recalled it getting a brief mention here: Colton, "Avoiding mean-square error bias in Designed experiments". On p.3 "Another inappropriate model-reduction technique can result in a positive bias in the LOF-error component" et seq. $\endgroup$
    – Scortchi ♦
    Commented Oct 25, 2013 at 19:26

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.