5
$\begingroup$

I have carried out a stepwise logistic regression in JMP. Then (using the proper button in the program window), I have chosen to build a nominal logistic regression model using (only) the variables identified by the stepwise procedure. Anyhow, comparing the summary tables of the stepwise regression and the nominal one, I have recognized that the regression coefficients are not the same, and also the p-values are not the same. There is even a variable which changes from a p-value of 0.02 to a p-value of 0.19 (much greater that 0.10, the threshold value I have chosen before stepwise procedure to retain variables in the model!

How is it possible?

I could use the values in the stepwise summary, but it does not contains any data allowing to build the confidence intervals. So, in suborder my question is: how can I calculate the confidence intervals using only the data reported in JMP stepwise regression summary?

Edit: I have recognized just a minute ago that the differences refer to categorical variables which have yield more than one significant comparison. For example, on stepwise regression details I read variable1 is included in the model three times (and passed three times to the nominal regression procedure): A-B versus C-D-E-F-G, C-D versus E-F-G, E-F versus G. Anyhow, such variable1 is reported only one time in regression summary, which cites only the first comparison (A-B versus C-D-E-F-G). It remains a mistery for me why.

$\endgroup$
5
  • 3
    $\begingroup$ This is a problem with stepwise regression and categorical variables. See this (very) recent question for more information: How should I handle categorical variables with multiple levels when doing backward elimination? $\endgroup$ – Matt Parker Jan 7 '11 at 17:01
  • $\begingroup$ I don't understand. If one categorical variable group (or partition, or what I have called "comparison") is not significant and all the groups have to be removed from the model, why in my case I find that one group is retained for nominal regression and the other two ones are discarded? $\endgroup$ – glassy Jan 7 '11 at 23:13
  • 1
    $\begingroup$ Because the algorithm is wrongly implemented and does not enforce hierarchy rules that it should. In SAS 9.3 you can do better with PROC GLMSELECT, see the hierarchy option. But you shouldn't use stepwise in any case, it's a flawed method. $\endgroup$ – Peter Flom Sep 30 '12 at 21:50
  • 1
    $\begingroup$ If any of the response or predictor factors in the stepwise regression have missing data for some rows the values of p-values, coefficient estimates, etc in the stepwise window can be incorrect. I'm not sure exactly what error is made in the JMP calculation but I have a dataset where I experiences this discrepancy, in it the stepwise calculations are based on my full dataset (n=72), but when I run the linear model on its own it is correctly based on only 50 samples and gives a different p-value, estimate, etc. So this may have been a factor in your issue, too. $\endgroup$ – DirtStats Oct 24 '16 at 19:25
  • 1
    $\begingroup$ @Peter Flom: do you want to post your comment(s) as an answer? Better to have a short answer than no answer at all. Anyone who has a better answer can post it. $\endgroup$ – kjetil b halvorsen Nov 23 '20 at 1:13
2
$\begingroup$

Because the algorithm is wrongly implemented and does not enforce hierarchy rules that it should. In SAS 9.3 you can do better with PROC GLMSELECT, see the hierarchy option. But you shouldn't use stepwise in any case, it's a flawed method.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.