# How to interpret statistics (Emax, D, U, Q, B, etc.) of bootstrap validation of logistic regression

I'm only a linguist, so my knowledge of statistics is very basic.

I fitted a logistic regression model with R (with lrm(formula, y=T, x=T)), and when I use the option validate(lrm), I get some statistics I don't really understand.

index.orig   training   test optimism index.corrected     n
Dxy           0.5984   0.6112  0.5461   0.0651          0.5333 40
R2            0.3258   0.3676  0.2929   0.0747          0.2511 40
Intercept     0.0000   0.0000 -0.0105   0.0105         -0.0105 40
Slope         1.0000   1.0000  0.8427   0.1573          0.8427 40
Emax          0.0000   0.0000  0.0399   0.0399          0.0399 40
D             0.2713   0.3176  0.2394   0.0782          0.1931 40
U            -0.0177  -0.0177  0.0092  -0.0269          0.0092 40
Q             0.2890   0.3353  0.2302   0.1051          0.1839 40
B             0.1864   0.1772  0.1972  -0.0201          0.2064 40
g             1.4632   1.6642  1.3460   0.3182          1.1449 40
gp            0.2840   0.3011  0.2703   0.0308          0.2532 40


I don't really understand most of that. I think R2 and Dxy are supposed to be statistics of how good the predictors are, but I'm not sure how I should interpret the values, does the corrected Dyx = 0.651 mean that there is a strong correlation, while the corrected R2 = 0.0747 means that the correlation is very weak? I think the model is overfitted, but I'm not sure if I'm right.

Also, the other statistics are totally strange to me. What are Emax, D, U, Q, B, g, and gp?

-

You need to consider several things. First, when you get into a new field it is advisable to study the field before using its methods. You probably need to take several stat courses or read several stat books to be able to make the right choices and to make the right interpretations. Second, you did not specify the model you used. Did it allow continuous predictors to be nonlinear? Did you fully pre-specify the model as the R rms package's validate function assumes when you don't specify bw=TRUE? In other words did you do any model selection that the bootstrap validation needs to re-run at each bootstrap re-sample? In terms of interpreting the output, see rms course notes at http://biostat.mc.vanderbilt.edu/CourseBios330; click on Handouts. Then read http://biostat.mc.vanderbilt.edu/wiki/pub/Main/FrankHarrell/logistCal.pdf. Also, run validate with B=200.
The data are seldom able to reliably tell you which factors are truly correlated with the outcome. But besides that, resampling model validation us ruined if you pretend that the model after deleting "insignificant" variables (at what $\alpha$ level?) was the model at the beginning. The result will be an inflation of the model's value, along with all the tremendous damage that stepwise variable selection does to the model. You have to let the resampling procedure run the same $Y$-related analyses repeatedly. Specify all candidate variables to validate with bw=TRUE + stopping rule. – Frank Harrell Mar 31 '13 at 13:29