3
$\begingroup$

For a given cut-point in a prediction model or score, the mean of sensitivity and specificity equals the AUC. I've read that and I have observed this empirically.

How can I prove this?

$\endgroup$
4
  • $\begingroup$ Where did you read that? How was it phrased exactly? Is it any cut-point, or a specific one? To me this sounds very wrong unless "equals" was actually "is correlated with" or something like this. $\endgroup$
    – Calimo
    Dec 3 '19 at 18:05
  • $\begingroup$ I found it somewhere online. I tested it with some data sets and found it to be true (in SAS). $\endgroup$ Dec 3 '19 at 20:47
  • $\begingroup$ This makes no sense at all. Either we have a very different definition of "equal" or your curve is quite special. That's why I asked for clarification. $\endgroup$
    – Calimo
    Dec 4 '19 at 6:23
  • $\begingroup$ Upon further inspection, my data appear to to show this result somewhat by chance but the mean of sens and spec is an approximation of the AUC. $\endgroup$ Dec 4 '19 at 17:04
4
$\begingroup$

The mean of sensitivity and specificity IS EQUAL to the AUC for a given cut-point

The ROC of a single cut-point looks like this: enter image description here

The area under this curve can be calculated geometrically using the area of the a rectangle (B) and two triangles (A and C).

$AUC = A + B + C$

$A = \frac{(1 - spec)\times \ sens}{2}$

$B = sens\times\ spec$

$C = \frac{spec \times(1 - sens)}{2}$

$= \frac{(1 - spec)\times \ sens}{2} + sens\times\ spec + \frac{spec \times(1 - sens)}{2}$

$= \frac{sens - sens \times \ spec}{2} + sens \times \ spec + \frac{spec - sens \times spec}{2}$

$= \frac{sens + spec}{2}$

$= the \ mean \ of \ sens \ and \ spec$

In R, the function pROC::auc only approximates this AUC using the trapezoidal rule. The simulation below shows that the approximation is very close.

This simulation uses a predictor and an outcome variable that are not correlated.

library(caret)
library(pROC)

nSim= 2000
results= rep(NA, nSim)
diff= rep(NA, nSim)
for (i in 1:nSim) {
  #generate "predictor" and "truth" data set from bernoulli distribution
  ds= as.data.frame(cbind(rep(NA, 500), rep(NA, 500)))
  colnames(ds)= c("predictor", "truth")
  ds$predictor= rbinom(500, 1, .75)
  ds$truth= rbinom(500, 1, .75)

  # calculate sens., spec., and auc
  auc= auc(predictor ~ truth, data= ds)
  sens= sensitivity(as.factor(ds$predictor), as.factor(ds$truth))
  spec= specificity(as.factor(ds$predictor), as.factor(ds$truth))
  meanSensSpec= mean(c(spec, sens))

  # Compare the mean of sens. and spec. to the auc
  results[i]= round(meanSensSpec, 4) == round(auc, 4)
  diff[i]= abs(auc-meanSensSpec) 
}

# Proportion of simulations where AUC = mean of sens and spec (rounded to 4 decimal places) with 95% conf. int.
mean(results)
mean(results)+c(-1,1)*qnorm(1-0.05/2)*sqrt((mean(results)/(1-mean(results)))/nSim)
# Absolute diifference between aUC and the mean of sens and spec with 95% conf. int.
mean(diff)
mean(diff)+c(-1,1)*qt(1-0.5/2, nSim-1)*sd(diff)/sqrt(nSim)

The AUC and the mean of sens and spec (all rounded to the 4 decimal place) are only equal about 10% of the time.

# Proportion of simulations where AUC = mean of sens and spec (rounded to 4 decimal places) with 95% conf. int.
> mean(results)
[1] 0.0955
> mean(results)+c(-1,1)*qnorm(1-0.05/2)*sqrt((mean(results)/(1-mean(results)))/nSim)
[1] 0.08125933 0.10974067

Probably more informative, the absolute difference between the AUC (rounded to decimal places — that is how the pROC package presents it) and the mean of sens and spec is about 0.0010.

> # Absolute diifference between aUC and the mean of sens and spec with 95% conf. int.
> mean(diff)
[1] 0.0009904823
> mean(diff)+c(-1,1)*qt(1-0.5/2, nSim-1)*sd(diff)/sqrt(nSim)
[1] 0.0009725486 0.0010084160
$\endgroup$
0
$\begingroup$

AUC is area under curve plotted on 1-specificity(x axis) against sensitivity on y axis. So for AUC there is a trade-off between accuracy versus the error you are willing to make. The left corner of the curve where we have the maximum sensitivity and least specificity. So intuitively, if you take the average of accuracy versus error it is most likely to resemble the AUC.

$\endgroup$
1
  • $\begingroup$ Yes, it makes sense intuitively but I can't get the math to work out. $\endgroup$ Dec 3 '19 at 20:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.