0
$\begingroup$

I have run a stepwise regression and found that some of the selected variables are not significant yet in a multiple regression with all variables included in the model those variables were significant. What can be the cause of this and which results would you recommend that I use?

Improve this question
$\endgroup$
1
  • $\begingroup$ Are they non-significant at all steps? $\endgroup$
    – Bioinformatics
    Mar 7, 2013 at 13:23

3 Answers 3

Reset to default
6
$\begingroup$

Is that possible? Yes. Stepwise regression pursues to maximize overall (joint) prediction by the variables left in the model while attempting to minimize their number. Because variables usually intercorrelate, their relations are complex and the significance level of the variable in the model after removing some other variables from it can change dramatically and in counter-intuitive direction. Variables in a regressional model not only compete for prediction - some of them at times behave "peculiarly" (e.g. so called suppressor variables). Anyway, it is possible for a stepwise regression to keep a variable in a model even if it is non-significant predictor in this model. It is kept because it serves good for the overall prediction - in the specific company of the other predictors in the model. Such non-significant variable obviously shouldn't be interpreted as "predictor" or "influencer", its role is, in a sense, consolidating. You may try to throw away it and see how much the overall prediction (such as R-square) drops; if it drops slightly for you, well you may go without that variable.

By the way, stepwise regression is heavily critisized nowadays. You might search this site for the relevant opinions and the discussions of alternative approaches.

Improve this answer
$\endgroup$
3
$\begingroup$

Because stepwise regression is an invalid statistical technique, its answers can and do disagree from those from other approaches.

Improve this answer
$\endgroup$
-1
$\begingroup$

apart from reasons mentioned in previous answer, it also can be because stepwise regression algorithm adjusted p-values for multiple testing.

Improve this answer
$\endgroup$
2
  • 3
    $\begingroup$ Could you please clarify and/or support this statement? For instance, which stepwise algorithm do you know that applies such adjustment? $\endgroup$
    – whuber ♦
    Mar 7, 2013 at 15:46
  • $\begingroup$ whuber, in our greedy age all algorithms that perform a test multiple times should have at least optionally accounted for multuple comparisons. $\endgroup$
    – coulminer
    Dec 18, 2013 at 13:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.