1
$\begingroup$

When is it appropriate to use a backward method in regression? I have read that it is permitted for exploratory model-building, but I have also read negative things about it.

I am making a model about streaming. The 12 predictors I chose are based on theory. The enter method is not an option, because when I put all independent variables (sociodemographic and others) in the regression with the enter method, only one independent variable is significant. When I use this independent variable and add an independent variable (which was significant from backward selection), both are significant, which means that the enter method filters variables out of the regression that are significant.

  1. Can I use the backward method to test all variables first, then use the significant variables to rerun the regression with an enter method, or must I be consistent and use the backward method again? Or is the backward method out of the question?

  2. May I select independent variables that have a significant effect on the dependent variable according to the correlation matrix to run the regression with, in this case maybe, the enter method?

$\endgroup$
3
$\begingroup$

You write:

The 12 predictors I chose are based on theory. The enter method is not an option, because when I put all independent variables (sociodemographic and others) in the regression with the enter method, only one independent variable is significant.

If the 12 predictors are based on theory and the sample size is large enough to include all 12, and there isn't collinearity or other problems, then I recommend including all 12.

If theory suggests a large value and you get a small one, that is interesting.

If you really want to, you could re-define the null from "no effect" to an effect size based on theoretical values.

$\endgroup$
  • 1
    $\begingroup$ They are not entirely based on theory. These predictors have not been tested before and the dependent variabele is new as well. The predictors are variables I suggest that will have an effect. I chose one predictor that came from literature. The sample size for the particular question of the dependent variable is 68 (entire sample size is 259). The correlations between variables exists but it not higher than ,500. Is it ok to use the backward-method in this case? $\endgroup$ – June May 20 '14 at 6:49
  • $\begingroup$ Backward estimation is rarely a good idea, nor are forward or stepwise. See e.g. Harrell Regression Modeling Strategies. Is your sample size 68 or 259? $\endgroup$ – Peter Flom - Reinstate Monica May 20 '14 at 10:07
1
$\begingroup$

What do you mean by "significant"?

In forward methods, i.e. when you add variables, the starting deviance is large so using nets with a large mesh size is recommended. In SAS, a p-value lesser than 0.50 is enough.

In backward methods the starting deviance is low, so being restrictive is better.

You could also try the stepwise method which is a modification of the forward-method and differs in that variables already in the model do not necessarily stay there: a combination of forward and backward.

$\endgroup$
  • 2
    $\begingroup$ Stepwise regression is probably a dangerous suggestion here. See "Algorithms for automatic model selection". $\endgroup$ – Nick Stauner May 19 '14 at 22:42
  • 1
    $\begingroup$ +1 to both Sergio and Nick. Stepwise regression while (very) often misused is not the devil and should be mentioned in the context of variable/model selection (possibly with a good deal of cross-validation). $\endgroup$ – usεr11852 says Reinstate Monic May 20 '14 at 0:27
1
$\begingroup$

You should start by figuring out what question you are trying to answer. My personal epiphany came when I realized that I had no idea what question is answered by stepwise methods, and whatever question it is was not any I was interested in asking.

If you just want a predictive model then why not use all 12? If you are interested how a given variable or subset of variables influences the response then why would you consider an automated method that may or may not include those variables? instead look at comparing full and reduced models of interest.

If you need a more parsimonious model then you may want to look into lasso, ridge, lars, or elaticnet methods instead.

One thing to consider trying for your own edification is to fit your model using the stepwise methods, then delete the 1st data point and refit, then put the 1st point back in and take out the second, proceed though all the data points this way and compare the models. If you end up with the same variables in each of the models (and the slope estimates are similar) then you can be fairly confident that you lucked out and found a decent model. But if the different models vary widely from each other then you should ask yourself why should you trust a method that would give such a different answer from such a small change.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.