7
$\begingroup$

I'm trying to make overlaid ROC curves to represent successive improvements in model performance when particular predictors are added one at a time to the model. I want one ROC curve for each of about 5 nested models (which I will define manually), all overlaid in one plot. For example:

    #outcome var
    y = c(rep(0,50), rep(1, 50))

    #predictors
    x1 = y + rnorm(100, sd = 1)
    x2 = y + rnorm(100, sd = 4)

    #correlations of predictors with outcome
    cor(x1, y)
    cor(x2, y)

    library(Epi)
    ROC(form = y ~ x1, plot = "ROC)
    ROC(form = y ~ x1 + x2, plot = "ROC")

I'd want the two ROC curves on the same plot (and ideally without the distracting model info in the background). Any ggplot/graphics gurus willing to lend a hand?

Improve this question
$\endgroup$

2 Answers 2

Reset to default
3
$\begingroup$

The caTools package provides the colAUC function. Use it and set the plotROC argument to TRUE. I have been satisfied with the graphs it produces.

Improve this answer
$\endgroup$
5
  • $\begingroup$ Is it possible to modify this function so I can plot ROC curves for multivariate models? So far I have only been able to overlay univariate ROC curves. What I really want is to compare nested models. Thanks! $\endgroup$
    – half-pass
    May 25, 2012 at 3:51
  • $\begingroup$ As long as the outcome is univariate/Bernoulli then sure. Just pass a matrix or data-frame to the X argument with each column representing the predictions from a different model. So matching your above example: colAUC(X=cbind(x1,x2),y=y,plotROC=TRUE) $\endgroup$
    – Shea Parkes
    May 25, 2012 at 14:52
  • $\begingroup$ I read your comment again and am not sure I fully comprehended your question. I am assuming you want to compare the predictions from nested models. To do that you just want the final prediction from each model, not the individual contributions to that prediction from each feature. $\endgroup$
    – Shea Parkes
    May 25, 2012 at 14:54
  • $\begingroup$ Yes, I do want to compare final predictions from nested models. But when I use colAUC on the example data above, I ended up with one ROC curve for y ~ x1 and one for y ~ x2. Instead I want overlaid curves for y ~ x1 and y ~ x1 + x2. Thanks. $\endgroup$
    – half-pass
    May 28, 2012 at 0:49
  • $\begingroup$ You would just need to build out the columns like this then colAUC(X=data.frame(fit1=x1,fit2=x1+x2),y=y,plotROC=TRUE) $\endgroup$
    – Shea Parkes
    May 28, 2012 at 16:44
3
$\begingroup$

If you'd like to overlay the ROC curves over each other, you can use the roc function from the pROC R package to get the sensitivity and specificity values and plot them out manually, #outcome var y = c(rep(0,50), rep(1, 50))

#predictors
x1 = y + rnorm(100, sd = 1)
x2 = y + rnorm(100, sd = 4)


model1 = glm(y ~ x1, family = binomial())
pred1 = predict(model1)
model2 = glm(y ~ x1 + x2, family = binomial())
pred2 = predict(model2)

library(pROC)
roc1 = roc(y, pred1)
roc2 = roc(y, pred2)

Specificity and Sensitivity Values

> str(roc1)
List of 15
\$ percent           : logi FALSE
\$ sensitivities     : num [1:101] 1 1 0.98 0.98 0.98 0.98 0.98 0.98 0.96...
\$ specificities     : num [1:101] 0 0.02 0.02 0.04 0.06 0.08 0.1 0.12 0.12 
 ...

or use the plot function as

plot(roc1, col = 1, lty = 2, main = "ROC")
plot(roc2, col = 4, lty = 3, add = TRUE)

Two ROC plots

Also, there is also the pROC::ggroc function for ggplot2 plotting abilities.

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ Just for information, you have to be careful with the package pROC, the default behaviour is to inverse the results if you get a negative value so your plot will also be inverted. It will only give you a warning. You have to set the direction option $\endgroup$
    – R. Prost
    Jan 25, 2018 at 13:21
  • 1
    $\begingroup$ direction > in which direction to make the comparison? “auto” (default): automatically define in which group the median is higher and take the direction accordingly. “>”: if the predictor values for the control group are higher than the values of the case group (controls > t >= cases). “<”: if the predictor values for the control group are lower or equal than the values of the case group (controls < t <= cases). to avoid it you should set it to ">" myAUC <- roc(Target ~ variable, dataset, direction = ">") $\endgroup$
    – R. Prost
    Jan 25, 2018 at 13:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.