Turns out I wrote a medium article just for that! Here it is :
https://medium.com/@nathanaim/mathematics-behind-roc-auc-interpretation-e4e6f202a015
TL;DR : to go through the end of the demonstration, one needs to use the convolution theorem.
If you don't want to change sites, here is the full trick. We want to show that, for a given binary classifier, :
$$ROC-AUC = P\left(X_1>X_0\right) = P\left(X_1-X_0>0\right)$$
where :
- X₁ is a continuous random variable giving the “score” output by our binary classifier for a randomly chosen positive sample
- X₀ is a continuous random variable giving the “score” output by our binary classifier for a randomly chosen negative sample
Definitions and preliminary results
First, some definitions :
- Let X₁ and X₀ be defined as above
- Let f₁ and f₀ be, respectively, the density function of X₁ and X₀
- Let F₁ and F₀ be, respectively, the repartition function of X₁ and X₀
- True Positive Rate (TPR) and False Positive Rate (FPR) have their usual meaning, i.e. :
$$TPR=\frac{TP}{P}\:\:\,FPR=\frac{FP}{N}$$
We can already observe that, for a classifier threshold T, a randomly chosen positive sample would be correctly classified (true positive) if X₁>T. So, for a randomly chosen positive sample, the probability of correctly classifying it is P(X₁>T). By definition of the TPR, it corresponds to the probability of correctly classifying a randomly chosen positive sample, so TPR(T) = P(X₁>T) = 1- P(X₁⩽ T) = 1-F₁(T). (1)
This also means, by definition of the density function, that :
$$TPR(T) = \int\limits_{T}^{+\infty} f_1(x)\: \mathrm{d}x$$
Similarly, we can show that FPR(T) = 1- F₀(T) (2) Demonstration
Now let’s dig into the calculus!
By definition of the ROC, we have that :
$$ROC-AUC = \int\limits_0^1 TPR(FPR)\: \mathrm{d}FPR$$ $$= \int\limits_0^1 TPR(FPR^{-1}(x))\: \mathrm{d}x$$
By using this change in variable :
$$T=FPR^{-1}(x)\iff\ x=FPR(T)$$
the integral becomes :
$$\int\limits_{+\infty}^{-\infty} TPR(T) \times FPR'(T)\: \mathrm{d}T$$
Now, thanks to (2) we know that we can express this integral as :
$$\int\limits_{+\infty}^{-\infty} TPR(T) \times (-f_0(T))\: \mathrm{d}T = \int\limits_{-\infty}^{+\infty} TPR(T) \times f_0(T)\: \mathrm{d}T$$
Thanks to (1) we know that this can be expressed as :
$$\int\limits_{-\infty}^{+\infty} \int\limits_{T}^{+\infty} f_1(x)\: \mathrm{d}x \times f_0(T)\: \mathrm{d}T$$
By using this change in variable for the inner integral :
$$v=x-T$$
the integral becomes :
$$\int\limits_{-\infty}^{+\infty} \int\limits_{0}^{+\infty} f_1(v+T)\: \mathrm{d}v \times f_0(T)\: \mathrm{d}T$$ $$= \int\limits_{0}^{+\infty} \int\limits_{-\infty}^{+\infty} f_0(T)\: \mathrm{d}T \times \: f_1(v+T)\: \mathrm{d}v$$
and by using this change in variable for the inner integral :
$$u=v+T$$
it becomes :
$$\int\limits_{0}^{+\infty} \int\limits_{-\infty}^{+\infty} f_1(u)\: \times f_0(u-v)\: \mathrm{d}u \: \mathrm{d}v$$
Do you get where we’re going? Yes, right to the convolution theorem! First, let’s point out that since f₀(t) is a density function of X₀, f₀(-t) is a density function of (-X₀). Then, according to the convolution theorem and assuming the convergence, a density of X₁- X₀=X₁+(- X₀) is :
$$\int\limits_{-\infty}^{+\infty} f_1(u)\: \times f_0(u-v)\: \mathrm{d}u$$
This means that :
$$P\left(X_1>X_0\right)=P\left(X_1-X_0>0\right)$$ $$=\int\limits_{0}^{+\infty} \int\limits_{-\infty}^{+\infty} f_1(u)\: \times f_0(u-v)\: \mathrm{d}u \: \mathrm{d}v$$
And eventually we have that :
$$P\left(X_1>X_0\right) = ROC - AUC$$
Thanks for reading this far! Hope I helped :)