I'm trying to make overlaid ROC curves to represent successive improvements in model performance when particular predictors are added one at a time to the model. I want one ROC curve for each of about 5 nested models (which I will define manually), all overlaid in one plot. For example:

    #outcome var
    y = c(rep(0,50), rep(1, 50))

    x1 = y + rnorm(100, sd = 1)
    x2 = y + rnorm(100, sd = 4)

    #correlations of predictors with outcome
    cor(x1, y)
    cor(x2, y)

    ROC(form = y ~ x1, plot = "ROC)
    ROC(form = y ~ x1 + x2, plot = "ROC")

I'd want the two ROC curves on the same plot (and ideally without the distracting model info in the background). Any ggplot/graphics gurus willing to lend a hand?


The caTools package provides the colAUC function. Use it and set the plotROC argument to TRUE. I have been satisfied with the graphs it produces.

  • $\begingroup$ Is it possible to modify this function so I can plot ROC curves for multivariate models? So far I have only been able to overlay univariate ROC curves. What I really want is to compare nested models. Thanks! $\endgroup$ – half-pass May 25 '12 at 3:51
  • $\begingroup$ As long as the outcome is univariate/Bernoulli then sure. Just pass a matrix or data-frame to the X argument with each column representing the predictions from a different model. So matching your above example: colAUC(X=cbind(x1,x2),y=y,plotROC=TRUE) $\endgroup$ – Shea Parkes May 25 '12 at 14:52
  • $\begingroup$ I read your comment again and am not sure I fully comprehended your question. I am assuming you want to compare the predictions from nested models. To do that you just want the final prediction from each model, not the individual contributions to that prediction from each feature. $\endgroup$ – Shea Parkes May 25 '12 at 14:54
  • $\begingroup$ Yes, I do want to compare final predictions from nested models. But when I use colAUC on the example data above, I ended up with one ROC curve for y ~ x1 and one for y ~ x2. Instead I want overlaid curves for y ~ x1 and y ~ x1 + x2. Thanks. $\endgroup$ – half-pass May 28 '12 at 0:49
  • $\begingroup$ You would just need to build out the columns like this then colAUC(X=data.frame(fit1=x1,fit2=x1+x2),y=y,plotROC=TRUE) $\endgroup$ – Shea Parkes May 28 '12 at 16:44

If you'd like to overlay the ROC curves over each other, you can use the roc function from the pROC R package to get the sensitivity and specificity values and plot them out manually, #outcome var y = c(rep(0,50), rep(1, 50))

x1 = y + rnorm(100, sd = 1)
x2 = y + rnorm(100, sd = 4)

model1 = glm(y ~ x1, family = binomial())
pred1 = predict(model1)
model2 = glm(y ~ x1 + x2, family = binomial())
pred2 = predict(model2)

roc1 = roc(y, pred1)
roc2 = roc(y, pred2)

Specificity and Sensitivity Values

> str(roc1)
List of 15
\$ percent           : logi FALSE
\$ sensitivities     : num [1:101] 1 1 0.98 0.98 0.98 0.98 0.98 0.98 0.96...
\$ specificities     : num [1:101] 0 0.02 0.02 0.04 0.06 0.08 0.1 0.12 0.12 

or use the plot function as

plot(roc1, col = 1, lty = 2, main = "ROC")
plot(roc2, col = 4, lty = 3, add = TRUE)

Two ROC plots

Also, there is also the pROC::ggroc function for ggplot2 plotting abilities.

  • $\begingroup$ Just for information, you have to be careful with the package pROC, the default behaviour is to inverse the results if you get a negative value so your plot will also be inverted. It will only give you a warning. You have to set the direction option $\endgroup$ – R. Prost Jan 25 '18 at 13:21
  • $\begingroup$ direction > in which direction to make the comparison? “auto” (default): automatically define in which group the median is higher and take the direction accordingly. “>”: if the predictor values for the control group are higher than the values of the case group (controls > t >= cases). “<”: if the predictor values for the control group are lower or equal than the values of the case group (controls < t <= cases). to avoid it you should set it to ">" myAUC <- roc(Target ~ variable, dataset, direction = ">") $\endgroup$ – R. Prost Jan 25 '18 at 13:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.