If this answer on Quora is correct then I think I understand what concordance probability is. However, I also find that this answer on StackExchange provides the formula of concordance probability that is a little bit different from the one on Quora: It include the counts of tied pairs with weight $ 0.5 $. Which one should I trust?

I also understand AUC stands for Area Under a Curve and how to compute it... visually.

What I do not understand is how AUC equal concordance probability?

I have just found my question is the same as this answered question on StackExchange.


marked as duplicate by Sycorax, gung Sep 16 '17 at 1:36

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

  • $\begingroup$ What do you mean by computing it visually? You can just sweep your threshold across the range to get the points on the curve. Then compute the area underneath like you would compute an integral. $\endgroup$ – Aaron Apr 6 '17 at 18:25
  • $\begingroup$ @Aaron That is what I meant by saying "compute it visually"... I do not know of a shorter formula to calculate AUC except concordance probability. I'm sorry for my incomprehensible words. $\endgroup$ – ntvy95 Apr 7 '17 at 0:38
  • $\begingroup$ The sklearn library computes it using the trapezoidal rule for integration scikit-learn.org/stable/modules/generated/… $\endgroup$ – Aaron Apr 7 '17 at 0:41
  • $\begingroup$ @Aaron Thank you very much for your suggestion! :) $\endgroup$ – ntvy95 Apr 7 '17 at 1:08

I finally have just figured it out. I'm not sure if my reasoning is correct so I will leave it here for anyone to point out the errors.

The discrete formula of AUC should be:

$$ \sum_{i} \frac{TP_{i}}{P}\frac{FP_{i+1}-FP_{i}}{N} $$

(borrowed from http://mlwiki.org/index.php/ROC_Analysis)

That is, I also happen to calculate the (sometimes maybe just approximate) number of concordance pairs in the numerator ($ TP_{i}(FP_{i+1}-FP_{i}) $) and the number of total pairs in the denominator ($ PN $).

For an illustration:

  • When $ i = 2 $, $ TP_{2} = 2 $ and $ FP_{3} - FP_{2} = 1$, that is we have found $ 2 $ possible concordance pairs at $ i = 2 $ (i.e. 1-3, 2-3).

  • When $ i = 6 $, $ TP_{6} = 5 $ and $ FP_{7} - FP_{6} = 1$, that is we have found more $ 6 $ possible concordance pairs at $ i = 6 $ (i.e. 1-7, 2-7, 4-7, 5-7, 6-7). The iterative process keeps going on.

  • ...

However, this formula of AUC does not count tied pairs with 0.5 weight according to this answer on StackExchange.

  • $\begingroup$ I have found that this answer on StackExchange with some modifications can cover the case counting tied pairs with weight $ 0.5 $. $\endgroup$ – ntvy95 Apr 7 '17 at 1:48

Please forgive the shoddy pictures.

I use basic integration to prove that Area Under ROC and % of Concordant pairs are the same. I am adding a description below each illustration to make the understanding clear.

Figure 1 contains the distribution of 0's and 1's , $x(c), y(c)$ respectively. The cutoff $c$ is represented on the X-axis.

Figure 2 contains the distribution of ROC curve. The sensitivity is $Y$ and the (1-specificity) is $X$

Illustration 1

In the above picture, we begin by counting the total number of concordant pairs. The number of concordant pairs, for a given $x(c)dc$ number of 1's, is given by $[x(c)dc]*[\int_0^c y(c)]$.

So, the total number of concordant pairs is found by integrating the above expression from 0 to 1, i.e. $\int_0^1[x(c)*[\int_0^c y(c)]]dc$

Proportion of concordant pairs is found by ${(\int_0^1[x(c)*[\int_0^c y(c)]]dc) \over ((\int_0^1 x(c)dc)*(\int_0^1 y(c)dc))} \tag{1}\label{1} $

Now we turn to the ROC plot. For a given cutoff $c$, we obtain the specificity ($1-X$) by ${\int_0^c y(c)dc \over \int_0^1 y(c)dc} $.

Illustration 2

We obtain $dX = {y(c)dc \over \int_0^1 y(c)dc} $. As $X$ varies from 0 to 1, $c$ varies from 1 to 0.

Similarly, we obtain the expression for Sensitivity. And we go on to get the area under ROC = $\frac{[\int_0^1[\int_0^1 x dc] y dc]}{(\int_0^1 x(c)dc)(\int_0^1 y(c)dc)} \tag{2}\label{2}$

Now we have to show that the numerators of (1) and (2) are the same. This can be easily shown by using integration by parts.

Illustration 3


Not the answer you're looking for? Browse other questions tagged or ask your own question.