How to calculate Area Under the Curve (AUC), or the c-statistic, by hand I am interested in calculating area under the curve (AUC), or the c-statistic, by hand for a binary logistic regression model.
For example, in the validation dataset, I have the true value for the dependent variable, retention (1 = retained; 0 = not retained), as well as a predicted retention status for each observation generated by my regression analysis using a model that was built using the training set (this will range from 0 to 1).
My initial thoughts were to identify the "correct" number of model classifications and simply divide the number of "correct" observations by the number of total observations to calculate the c-statistic. By "correct", if the true retention status of an observation = 1 and the predicted retention status is > 0.5 then that is a "correct" classification. Additionally, if the true retention status of an observation = 0 and the predicted retention status is < 0.5 then that is also a "correct" classification. I assume a "tie" would occur when the predicted value = 0.5, but that phenomenon does not occur in my validation dataset. On the other hand, "incorrect" classifications would be if the true retention status of an observation = 1 and the predicted retention status is < 0.5 or if the true retention status for an outcome = 0 and the predicted retention status is > 0.5. I am aware of TP, FP, FN, TN, but not aware of how to calculate the c-statistic given this information.
 A: Karl's post has a lot of excellent information.  But I have not yet seen in the past 20 years an example of an ROC curve that changed anyone's thinking in a good direction.  The only value of an ROC curve in my humble opinion is that its area happens to equal a very useful concordance probability.  The ROC curve itself tempts the reader to use cutoffs, which is bad statistical practice.
As far as manually calculating the $c$-index, make a plot with $Y=0,1$ on the $x$-axis and the continuous predictor or predicted probability that $Y=1$ on the $y$-axis.  If you connect every point with $Y=0$ with every point with $Y=1$, the proportion of the lines that have a positive slope is the concordance probability.
Any measures that have a denominator of $n$ in this setting are improper accuracy scoring rules and should be avoided.  This includes proportion classified correctly, sensitivity, and specificity.
For the R Hmisc package rcorr.cens function, print the entire result to see more information, especially a standard error.
A: Here is an alternative to the natural way of calculating AUC by simply using the trapezoidal rule to get the area under the ROC curve. 
The AUC is equal to the probability that a randomly sampled positive observation has a predicted probability (of being positive) greater than a randomly sampled negative observation. You can use this to calculate the AUC quite easily in any programming language by going through all the pairwise combinations of positive and negative observations. You could also randomly sample observations if the sample size was too large. If you want to calculate AUC using pen and paper, this might not be the best approach unless you have a very small sample/a lot of time. For example in R:
n <- 100L

x1 <- rnorm(n, 2.0, 0.5)
x2 <- rnorm(n, -1.0, 2)
y <- rbinom(n, 1L, plogis(-0.4 + 0.5 * x1 + 0.1 * x2))

mod <- glm(y ~ x1 + x2, "binomial")

probs <- predict(mod, type = "response")

combinations <- expand.grid(positiveProbs = probs[y == 1L], 
        negativeProbs = probs[y == 0L])

mean(combinations$positiveProbs > combinations$negativeProbs)
[1] 0.628723

We can verify using the pROC package:
library(pROC)
auc(y, probs)
Area under the curve: 0.6287

Using random sampling:
mean(sample(probs[y == 1L], 100000L, TRUE) > sample(probs[y == 0L], 100000L, TRUE))
[1] 0.62896

A: Have a look at this question: Understanding ROC curve
Here's how to build a ROC curve (from that question):
Drawing ROC curve
given a data set processed by your ranking classifier 


*

*rank test examples on decreasing score

*start in $(0, 0)$

*for each example $x$ (in the decreasing order)


*

*if $x$ is positive, move $1/\text{pos}$ up

*if $x$ is negative, move $1/\text{neg}$ right



where $\text{pos}$ and $\text{neg}$ are the fractions of positive and negative examples respectively. 
You can use this idea for manually calculating AUC ROC using the following algorithm: 
auc = 0.0
height = 0.0

for each training example x_i, y_i
  if y_i = 1.0:
    height = height + tpr
  else 
    auc = auc + height * fpr

return auc

This nice gif-animated picture should illustrate this process clearer 

A: *

*You have true value for observations.  

*Calculate posterior probability and then rank observations by this probability.  

*Assuming cut-off probability of $P$ and number of observations $N$:
$$\frac{\text{Sum of true ranks}-0.5PN(PN+1)}{PN(N-PN)}$$

