# How to interpret this criterion of logistic regression result

I run a backward variable selection logistic regression and find out the SAS program selected 12 variables and give me the output like this:

It is funny that my configuration of the SAS backward selection is to make the SC(SBC) lower. Overall, the result looks like the SC result suppose that intercept with covariates is worse. But other two guys AIC and -2logL supports that Intercept with corvar is better model.

May you guys make some judge ment on this result and test output?

• Have you checked out the answers to this question - stats.stackexchange.com/questions/3984/aic-and-sc-value? – Peter Ellis Mar 24 '12 at 0:54
• @PeterEllis, I just checked. I am doing predicting model, so my model is good with those covariate....? And I　find a page interpreting that aic and sc are kind of "indicators" but checking the model with -2 log likelihood and degree of freedom improvement is better decision making. ats.ucla.edu/stat/sas/library/comp_repeated.htm – Wenhao.SHE Mar 24 '12 at 1:38
• @Wenhao.SHE I would strongly suggest that you, as the human, select the variables that go into the model based on logic and theory. The best a machine can do is data mine. Stepwise selection methods are notorious for capitalizing on sample idiosyncrasies. As for your fit statistics, have you considered looking at something like the area under the ROC curve (c-statistic)? I find it's a much more intuitive measure of model fit. – Will Mar 24 '12 at 2:17
• @Will, I just post the ROC curve. May you have a look?> I try to predict whether tomorrow a forex pair is up or down. as you can see, the variable is frequecy of key trading related words. It makes sense for human, right? May you give more comments after viewing the roc? Thanks – Wenhao.SHE Mar 24 '12 at 2:23
• @Wenhao.SHE, I see. There isn't any sort of theory here, this really is just data mining. I would use the AIC in this situation. As to the c-statistic, a value of .5 means your model's predictions are no better than chance while a value of 1 would mean perfect prediction. It's just a nice intuitive way for humans to pick one model over another. For your purposes it probably isn't as useful as the AIC. This isn't my subject area but I would hope for a c stat higher than .75. – Will Mar 24 '12 at 3:06

Looking at the SC in conjunction with -2logL is suggesting that your model is overparametised. That is, you can obtain a simpler model with less parameters but still a good fit to the data. You will get a better SC if you set the slstay= option equal to the quantile of a chi-square with 1 degree of freedom at the difference between SC and -2LogL for the intercept only model. For your output we have a difference of $447.224-440.919=6.305$ (from which I infer your sample size is $\exp(6.305)=547$). Now we set the p-value equal to $Pr(\chi^2_1>6.305)=0.012$ (more generally set it to $Pr(\chi^2_1>\log(n))$. This ensures that every model selection step will decrease the SC (approximately) without having to manually re-fit each model. If you haven't specified the slentry= option then it defaults to $0.05$. Thus my prediction is that you have some variables with p-values less than $0.05$ but greater than $0.012$.