# How does “stepwise regression” work?

I used the following R code to fit a probit model:

p1 <- glm(natijeh ~ ., family=binomial(probit), data=data1)
stepwise(p1, direction='backward/forward', criterion='BIC')


I want to know what does stepwise and backward/forward do exactly and how select the variables?

• Some comments by Frank Harrell (stats.stackexchange.com/users/4253/frank-harrell) on why stepwise regression is bad: stata.com/support/faqs/statistics/stepwise-regression-problems – user25658 Sep 7 '13 at 17:24
• In addition to BabakP's links, have also a look at this post from the site. – COOLSerdash Sep 7 '13 at 18:32
• Yet another post about problems with stepwise (and backward and forward as well) is a paper I wrote with David Cassell: Stopping Stepwise – Peter Flom Sep 7 '13 at 20:35
• @PeterFlom, in order to reference this paper, I am having some problems understanding the proper citation. Could you please list it here? Thanks. – doug.numbers May 23 '14 at 22:24
• @doug.numbers It was presented various places and published as part of conference proceedings. If you Google "Flom, Cassell, Stepwise" you'll get places it was presented and you can format it however you format citations to published presentations. – Peter Flom May 23 '14 at 22:28

Principle of stepwise selection

1. You fit a model with all variables you wish. This is your current best model.
2. You remove one variable (or add one, among variable not used in the current best model), and for each one, you fit the new model, and you compare them with each over and with the original one, according to BIC (or any other criterion, such as AIC). You get another "current best model".

You repeat 2. until there no reduction of BIC. You have only a local minimum of BIC, which means you may not get the best model among all possible choices of subsets of variables. But anyway, there are usually too many of them, so this is a way to optimize a bit, without too much work.

By specifying forward you are telling R that you would like to start with the simplest model (i.e., one covariate) and then add one covariate one at a time keeping only the ones that result in an improvement to the models BIC.
By specifying backward you are telling R that you want to start with the full model (i.e., the model with all the covariates) and then drop covariates, one ata time, that result in an improvement in the BIC.