How to perform cross validation of a model predicting categorical variable in SAS?

Question

I have a prediction outcome that is dichotomous while all other predictor variables are binary and categorical.

How can I perform cross validation in SAS and see the ROC curve?

I need to create bayesian network.

Answer 1

In the output statement in proc logistic you can request cross-validated predictions (I believe they are leave one out predictions). There is also an option to produce an ROC curve.

If you need to go further you're going to have to create training and test data sets in a data step, run the model, save predicted probabilities, then predict on the test set in another data step.

Answer 2

I'm not able to recomend you a better response than this link, always having into account that these are straightforward solutions:

ROC analysis using validation data and crossvalidation

This paper describes solutions that way :

proc logistic data=alldata;
    model y(event="1") = entry;
    output out=preds predprobs=crossvalidate;
    run;

  proc logistic data=preds;
    model y(event="1") = entry;
    roc pred=xp_1;
    roccontrast
    run;

See details in the link and how this is built up. Other aproximations have to be done by contructing yourself the CV sets through data steps and smapling and later calculating the necessary parameters for ROC curves, which i would do generating aggregate counts and means after the cross validation training and validation steps when possible.