For a given cut-point in a prediction model or score, the mean of sensitivity and specificity equals the AUC. I've read that and I have observed this empirically.
How can I prove this?
For a given cut-point in a prediction model or score, the mean of sensitivity and specificity equals the AUC. I've read that and I have observed this empirically.
How can I prove this?
The mean of sensitivity and specificity IS EQUAL to the AUC for a given cut-point
The ROC of a single cut-point looks like this:
The area under this curve can be calculated geometrically using the area of the a rectangle (B) and two triangles (A and C).
$AUC = A + B + C$
$A = \frac{(1 - spec)\times \ sens}{2}$
$B = sens\times\ spec$
$C = \frac{spec \times(1 - sens)}{2}$
$= \frac{(1 - spec)\times \ sens}{2} + sens\times\ spec + \frac{spec \times(1 - sens)}{2}$
$= \frac{sens - sens \times \ spec}{2} + sens \times \ spec + \frac{spec - sens \times spec}{2}$
$= \frac{sens + spec}{2}$
$= the \ mean \ of \ sens \ and \ spec$
In R, the function pROC::auc only approximates this AUC using the trapezoidal rule. The simulation below shows that the approximation is very close.
This simulation uses a predictor and an outcome variable that are not correlated.
library(caret)
library(pROC)
nSim= 2000
results= rep(NA, nSim)
diff= rep(NA, nSim)
for (i in 1:nSim) {
#generate "predictor" and "truth" data set from bernoulli distribution
ds= as.data.frame(cbind(rep(NA, 500), rep(NA, 500)))
colnames(ds)= c("predictor", "truth")
ds$predictor= rbinom(500, 1, .75)
ds$truth= rbinom(500, 1, .75)
# calculate sens., spec., and auc
auc= auc(predictor ~ truth, data= ds)
sens= sensitivity(as.factor(ds$predictor), as.factor(ds$truth))
spec= specificity(as.factor(ds$predictor), as.factor(ds$truth))
meanSensSpec= mean(c(spec, sens))
# Compare the mean of sens. and spec. to the auc
results[i]= round(meanSensSpec, 4) == round(auc, 4)
diff[i]= abs(auc-meanSensSpec)
}
# Proportion of simulations where AUC = mean of sens and spec (rounded to 4 decimal places) with 95% conf. int.
mean(results)
mean(results)+c(-1,1)*qnorm(1-0.05/2)*sqrt((mean(results)/(1-mean(results)))/nSim)
# Absolute diifference between aUC and the mean of sens and spec with 95% conf. int.
mean(diff)
mean(diff)+c(-1,1)*qt(1-0.5/2, nSim-1)*sd(diff)/sqrt(nSim)
The AUC and the mean of sens and spec (all rounded to the 4 decimal place) are only equal about 10% of the time.
# Proportion of simulations where AUC = mean of sens and spec (rounded to 4 decimal places) with 95% conf. int.
> mean(results)
[1] 0.0955
> mean(results)+c(-1,1)*qnorm(1-0.05/2)*sqrt((mean(results)/(1-mean(results)))/nSim)
[1] 0.08125933 0.10974067
Probably more informative, the absolute difference between the AUC (rounded to decimal places — that is how the pROC package presents it) and the mean of sens and spec is about 0.0010.
> # Absolute diifference between aUC and the mean of sens and spec with 95% conf. int.
> mean(diff)
[1] 0.0009904823
> mean(diff)+c(-1,1)*qt(1-0.5/2, nSim-1)*sd(diff)/sqrt(nSim)
[1] 0.0009725486 0.0010084160
AUC is area under curve plotted on 1-specificity(x axis) against sensitivity on y axis. So for AUC there is a trade-off between accuracy versus the error you are willing to make. The left corner of the curve where we have the maximum sensitivity and least specificity. So intuitively, if you take the average of accuracy versus error it is most likely to resemble the AUC.