How to select predictors in regression using backward method?

Question

When is it appropriate to use a backward method in regression? I have read that it is permitted for exploratory model-building, but I have also read negative things about it.

I am making a model about streaming. The 12 predictors I chose are based on theory. The enter method is not an option, because when I put all independent variables (sociodemographic and others) in the regression with the enter method, only one independent variable is significant. When I use this independent variable and add an independent variable (which was significant from backward selection), both are significant, which means that the enter method filters variables out of the regression that are significant.

1. Can I use the backward method to test all variables first, then use the significant variables to rerun the regression with an enter method, or must I be consistent and use the backward method again? Or is the backward method out of the question?

2. May I select independent variables that have a significant effect on the dependent variable according to the correlation matrix to run the regression with, in this case maybe, the enter method?

Answer 1

You write:

The 12 predictors I chose are based on theory. The enter method is not an option, because when I put all independent variables (sociodemographic and others) in the regression with the enter method, only one independent variable is significant.

If the 12 predictors are based on theory and the sample size is large enough to include all 12, and there isn't collinearity or other problems, then I recommend including all 12.

If theory suggests a large value and you get a small one, that is interesting.

If you really want to, you could re-define the null from "no effect" to an effect size based on theoretical values.

Answer 2

What do you mean by "significant"?

In forward methods, i.e. when you add variables, the starting deviance is large so using nets with a large mesh size is recommended. In SAS, a p-value lesser than 0.50 is enough.

In backward methods the starting deviance is low, so being restrictive is better.

You could also try the stepwise method which is a modification of the forward-method and differs in that variables already in the model do not necessarily stay there: a combination of forward and backward.

Answer 3

You should start by figuring out what question you are trying to answer. My personal epiphany came when I realized that I had no idea what question is answered by stepwise methods, and whatever question it is was not any I was interested in asking.

If you just want a predictive model then why not use all 12? If you are interested how a given variable or subset of variables influences the response then why would you consider an automated method that may or may not include those variables? instead look at comparing full and reduced models of interest.

If you need a more parsimonious model then you may want to look into lasso, ridge, lars, or elaticnet methods instead.

One thing to consider trying for your own edification is to fit your model using the stepwise methods, then delete the 1st data point and refit, then put the 1st point back in and take out the second, proceed though all the data points this way and compare the models. If you end up with the same variables in each of the models (and the slope estimates are similar) then you can be fairly confident that you lucked out and found a decent model. But if the different models vary widely from each other then you should ask yourself why should you trust a method that would give such a different answer from such a small change.