1
$\begingroup$

I'm using the model ReLogit from the package Zelig in R. ReLogit is a logistic regression for rare events data. After having estimated the model on the training set, I want to calculate the AUC (Area under the ROC Curve) of this model in the test set. How can I do this with this package?

$\endgroup$
2
$\begingroup$

I think the easiest way to do this is using the pROC package. In Zelig I think you can use rocplot or something to extract the statistic.

Let's say we estimate m1, then one way to calculate the AUC is:

library(pROC)
library(Zelig)

data(mid) # Data on Militarized Interstate Disputes from King & Zeng (2001)
m1<-relogit(conflict~.,mid)
auc(as.numeric(m1$model[1]>0),as.vector(fitted(m1)))

Area under the curve: 0.9235

And now we're at it, it's also easy to plot the ROC:

plot(roc(as.numeric(m1$model[1]>0),as.vector(fitted(m1))))

enter image description here

$\endgroup$
1
  • $\begingroup$ The curve doesn't help and the area is just the Wilcoxon-Mann-Whitney U-statistic (concordance probability). In R you can compute that with one line. See for example the Hmisc package's somers2 function. $\endgroup$ – Frank Harrell Nov 18 '15 at 20:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.