# Method to output statistical score when validating using bootstrap method on ordinal logistic regression model

I build an ordinal logistic regression model on my data.

Level  Meaning
2    Unusual Event
1    Non Emergency


As you can see, it is my ordinal response variable. 3 means large loss and Alert situation, etc. I build a very simple model, the graph below shows the result:

 proc logistic data = sasuser.Pacific_West_ outmodel=sasuser.Pacifici_West_model;;
model Numerical_Emergency(event = '1') = AGE;
run;


Everything looks fine. Now I would like to apply the BOOTSTRAP method to validate this model. What thing I would like to see is the test c value as ROC value. As you may see in the graph above, the c (ROC) value is 0.640. The basic idea of bootstrap is to generate new data set by random picking process from the data we used to train the model as we all know. I prefer to generate 1000 copies of data with same size of original data for building the model in graph above.

THE PROBLEM:

In the SAS Score statement, I just found outroc method which does not support the ordinal regression. Is there a simple procedure to help me to output the statistics about fitting of trained model on the validation dataset build by myself?

Thank you so much for your time. Have a nice day!

Wenhao SHE

-
 Where is the curve above? I am also not quite sure what you mean by using the bootstrap to validate the model. Do you mean that you want to repeat the logistic regression procedure on the bootstrap samples to see if the parameter estimates don't vary much and all stay significant (at least in most bootstrap samples)? – Michael Chernick May 4 '12 at 15:40 @MichaelChernick, Thank you for your reply. I do not want to rerun the logistic regression on generated bootstrap dataset. Instead, I want to apply the trained model from original dataset to the generated data. And check the variation of ROC value or range of ROC value. I do not know how to output. – Wenhao.SHE May 4 '12 at 15:51 I'm not sure I understand what you want to do, and as @MichaelChernick noted, you don't show a curve. I'm not sure what the curve would be. However, there is also the OUT = statement that lets you output many different statistics – Peter Flom May 4 '12 at 20:27 @PeterFlom Thank you for your reply. I do not want to draw the graphic of ROC. Simply say, I have a trained model which tells me the beta for variables. I would use these betas on prepared validation data set. From applying model, I prefer to have ROC from validation dataset which would be certainly different from the ROC of trained model on training data set. I would like to compare them.:) – Wenhao.SHE May 4 '12 at 21:46