1
$\begingroup$

I want to perform a regression and want an objective strategy to decide on which variables to include (so something besides theory or expert judgement). I am aware of stepwise backward and forward methods, in which the most significant variable is added or the least significant variable is removed up to the point where the model fit does no longer significantly improve.

I want to make a decision based on BIC, but I do not want to consider all possible combinations of variables (excluding polynomial terms and interactions). If I have $n$ variables, then this means I need to perform $(n) (n!)$ regressions (I think), which is too many.

I was hoping to find a stepwise approach for the BIC methods. The reason is simply to reduce the number of cases I need to check. Does there exist a stepwise algorithm for the BIC method? I could not find any stepwise approach that does not consider p-values.

$\endgroup$
5
  • $\begingroup$ You can easily adapt the standard stepwise approach to use BIC rather than p values. The bigger question is: why do you want to do this? This can work if your goal is predicting, but if you want to do hypothesis testing or inference on the final model, then this approach will render any tests invalid, just as any other stepwise method. $\endgroup$ Commented Apr 26, 2023 at 10:51
  • $\begingroup$ I want to do this, because my variables are not really direct measures of what I want to predict. For example, I want to explain education by wealth of the parents, but I do not have parental income as variable. However, I do have the value of the house the person grew up in. $\endgroup$
    – Xtiaan
    Commented Apr 26, 2023 at 11:14
  • $\begingroup$ I don't quite see how that is an argument for a stepwise approach. And it does sound like you do want to do inference, so please take a look at the relevant threads - any stepwise approach invalidates p values. $\endgroup$ Commented Apr 26, 2023 at 11:20
  • $\begingroup$ But how would one combine a stepwise approach with BIC, because BIC only says something about the entire model and not about specific variables. $\endgroup$
    – Xtiaan
    Commented Apr 26, 2023 at 11:21
  • 1
    $\begingroup$ If you really want to do this, you would start from a model $M$ and try adding (or removing) one variable at a time, fitting a new model $M'$ and recording the BIC values of all these new models. Then move to the one new model that has the lowest BIC, and iterate. If no model achieves a lower BIC than the current model $M$, then $M$ is your final model. It's completely analogous to adding variables based on their p values in a larger model. (And it's still not a good idea.) $\endgroup$ Commented Apr 26, 2023 at 11:30

1 Answer 1

0
$\begingroup$

BIC is one criterion for adding or removing a single variable in stepwise selection. There is the MASS::stepAIC function in R. Similar logic could be applied to BIC.

As is discussed in the comments, though, there are major issues with stepwise selection. Stepwise regression seems like it can be competitive for pure prediction problems, but it’s use requires some care, and there are major flaws when you do inference on stepwise-selected variables.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.