In a paper by Faraklas et al, the researchers create a Necrotizing Soft-Tissue Infection Mortality Risk Calculator. They use logistic regression to create a model with mortality from necrotizing soft-tissue infection as the main outcome and then calculate the area under the curve (AUC). They use the bootstrap method to find the "bootstrap optimism-corrected ROC area."

If I were to do this in R, how would it look like? The code I have been toying with looks something like below:


auc_calc <- function(data, indices, outcomes) {
  d <- data[indices,]
  # Using glm for logistic regression
  # Do I recreate the glm model for each dataset?
  fit <- glm(outcomes[indices,] ~ X1 + X2 + X3, data=d, family=binomial)
  fit.predict <- predict(fit, type="response")

  # Using ROCR to calculate AUC
  pred <- prediction(fit.predict, outcomes[indices,])
  perf <- performance(pred, "auc")

  # Returning the AUC

boot.results <- boot(data=my.data, statistic=auc_calc, R=10000, outcomes=my.outcomes)

Is this correct? Or am I doing something wrong - namely should I be passing in a glm model rather than recalculating it each time? As always thanks for the help.

  • 4
    $\begingroup$ Frank Harrell's rms package has functions for this task. Fit the model with fit <- lrm(outcomes ~ X1 + X2 + X3, data=my.data, x=TRUE, y=TRUE), then use bootstrap validation with validate(fit, B=1000). The output matrix includes the optimism corrected values, but only shows Somers' $D_{xy}$. However $\text{AUC} = 0.5 \cdot D_{xy} + 0.5$. $\endgroup$ – caracal Jun 10 '13 at 17:38
  • $\begingroup$ I would like to avoid using and relearning another package that would force me to rewrite what I have so far. Is there no way to do what I want using boot and ROCR? $\endgroup$ – oort Jun 11 '13 at 18:02
  • $\begingroup$ Have you ever found a solution to this? $\endgroup$ – enricoferrero Aug 22 '17 at 12:26
  • $\begingroup$ I emailed Harrell to ask why the validate does not output the C (auc) metric. Annoying. $\endgroup$ – CoderGuy123 Sep 21 '17 at 22:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.