5
$\begingroup$

I have carried out a stepwise logistic regression in JMP. Then (using the proper button in the program window), I have chosen to build a nominal logistic regression model using (only) the variables identified by the stepwise procedure. Anyhow, comparing the summary tables of the stepwise regression and the nominal one, I have recognized that the regression coefficients are not the same, and also the p-values are not the same. There is even a variable which changes from a p-value of 0.02 to a p-value of 0.19 (much greater that 0.10, the threshold value I have chosen before stepwise procedure to retain variables in the model!

How is it possible?

I could use the values in the stepwise summary, but it does not contains any data allowing to build the confidence intervals. So, in suborder my question is: how can I calculate the confidence intervals using only the data reported in JMP stepwise regression summary?

Edit: I have recognized just a minute ago that the differences refer to categorical variables which have yield more than one significant comparison. For example, on stepwise regression details I read variable1 is included in the model three times (and passed three times to the nominal regression procedure): A-B versus C-D-E-F-G, C-D versus E-F-G, E-F versus G. Anyhow, such variable1 is reported only one time in regression summary, which cites only the first comparison (A-B versus C-D-E-F-G). It remains a mistery for me why.

Improve this question
$\endgroup$
5
  • 3
    $\begingroup$ This is a problem with stepwise regression and categorical variables. See this (very) recent question for more information: How should I handle categorical variables with multiple levels when doing backward elimination? $\endgroup$
    – Matt Parker
    Commented Jan 7, 2011 at 17:01
  • $\begingroup$ I don't understand. If one categorical variable group (or partition, or what I have called "comparison") is not significant and all the groups have to be removed from the model, why in my case I find that one group is retained for nominal regression and the other two ones are discarded? $\endgroup$
    – glassy
    Commented Jan 7, 2011 at 23:13
  • 1
    $\begingroup$ Because the algorithm is wrongly implemented and does not enforce hierarchy rules that it should. In SAS 9.3 you can do better with PROC GLMSELECT, see the hierarchy option. But you shouldn't use stepwise in any case, it's a flawed method. $\endgroup$
    – Peter Flom
    Commented Sep 30, 2012 at 21:50
  • 1
    $\begingroup$ If any of the response or predictor factors in the stepwise regression have missing data for some rows the values of p-values, coefficient estimates, etc in the stepwise window can be incorrect. I'm not sure exactly what error is made in the JMP calculation but I have a dataset where I experiences this discrepancy, in it the stepwise calculations are based on my full dataset (n=72), but when I run the linear model on its own it is correctly based on only 50 samples and gives a different p-value, estimate, etc. So this may have been a factor in your issue, too. $\endgroup$
    – DirtStats
    Commented Oct 24, 2016 at 19:25
  • 1
    $\begingroup$ @Peter Flom: do you want to post your comment(s) as an answer? Better to have a short answer than no answer at all. Anyone who has a better answer can post it. $\endgroup$
    – kjetil b halvorsen ♦
    Commented Nov 23, 2020 at 1:13

1 Answer 1

Reset to default
3
$\begingroup$

Because the algorithm is wrongly implemented and does not enforce hierarchy rules that it should. In SAS 9.3 you can do better with PROC GLMSELECT, see the hierarchy option. But you shouldn't use stepwise in any case, it's a flawed method.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.