Derivation of closed-form ROC expression under binormal assumptions

Question

It's a known result that, under binormality assumptions, the area under the ROC curve (AUC) for a binary classifier has the following closed form. Formally, define the class conditional mean and variance for class $y \in \{0, 1\}$:

$$\mu_y = \mathbb{E}[X \mid Y = y]$$ $$\sigma^2_y = Var[X \mid Y = y]$$

Further assume $\mu_1 \geq \mu_0$. Then the AUC is given by

$$AUC = \Phi\left(\frac{\mu_1 - \mu_0}{\sqrt{\sigma_1^2 + \sigma_0^2}}\right)$$

where $\Phi$ is the Normal CDF. I recently encountered this form in Fong et. al. (2021). A search turns up variations on this form as well, for example Pan and Metz (1997) or Bandos, Guo and Gur (2018).

I'm having trouble coming up with the derivation. Here's what I've tried. First, I know false positive rate (FPR) and true positive rate (TPR), under the binormal assumption, are given at some threshold $\tau$ as

$$TPR(\tau) = 1 - \Phi\left(\frac{\tau - \mu_1}{\sigma_1}\right) = \Phi\left(\frac{\mu_1 - \tau}{\sigma_1}\right)$$ $$FPR(\tau) = 1 - \Phi\left(\frac{\tau - \mu_0}{\sigma_0}\right) = \Phi\left(\frac{\mu_0 - \tau}{\sigma_0}\right)$$ so that the AUC can be written as

$$\int_0^1 TPR(FPR^{-1}(x)) dx.$$

However, I'm getting stuck expanding that integral (or maybe this is not the right method to derive this). Substituting in the definition of TPR and FPR under our assumptions, I get something like

$$\int_0^1 TPR(FPR^{-1}(x)) dx= \int_0^1 \Phi\left(\frac{\mu_1 - FPR^{-1}(x)}{\sigma_1}\right) dx = \int_0^1 \Phi\left(\frac{\mu_1 - (\Phi^{-1}(1-x)\sigma_0 + \mu_0)}{\sigma_1}\right) dx.$$

At this point, I'm unsure how to proceed towards the expected closed form for the AUC. Any tips on how to proceed from here, or resources with the derivation (any form of the AUC will do)?

Answer 1

This comes directly from the derivation of the probabilistic interpretation of AUC. First, we know

$$AUC = \int_0^1 TPR(FPR^{-1}(x)) dx.$$

Through a change of variables $u := FPR^{-1}(x)$, we have

$$AUC = \int_{FPR^{-1}(0)}^{FPR^{-1}(1)} TPR(u) FPR'(u) du = \int_{\infty}^{-\infty} \Phi\left(\frac{\mu_1 - u}{\sigma_1}\right)f_0(u) du,$$

where $f_0$ is the density of the negative class (i.e. pdf of $\mathcal{N}(\mu_0, \sigma_0^2)$). We'll later use the notation $f_1$ for the density of the positive class. To be rigorous, implicitly, there's another substitution at this phase as $FPR'(u) = f_0\left(\frac{u - \mu_0}{\sigma_0}\right)$, but the term inside $f_0$ has derivative one, so the effect is an identity multiplication.

Then, we can write the CDF (noting the flipped bounds this time) as $$\int_{-\infty}^{\infty} \left(\int_{-\infty}^{\infty} \mathbf{1}[t < u] \cdot f_1(t) \;dt\right)\; f_0(u) du = \int_{-\infty}^{\infty} \int_{-\infty}^{\infty} \mathbf{1}[t - u < 0] \cdot f_0(u)f_1(t) \;du \;dt$$

which is the CDF for a difference of normal distributions. This distribution has variance $\sigma_0^2 + \sigma_1^2$, and mean $\mu_1 - \mu_0$, which yields

$$AUC = \Phi\left( \frac{\mu_1 - \mu_0}{\sqrt{\sigma_0^2 + \sigma_1^2}}\right)$$

as needed.

Answer 2

About the probabilistic interpretation of the AUC...

Assume $X$ is a continuous measurement used to determine whether an individual has binary condition $D$ ($D = 0$ or $1$) and that higher values of $X$ are more suggestive of $D = 1$: $$F_1(x) = P(X \le x|D=1)$$ $$F_0(x) = P(X \le x|D=0)$$ $$f_0(x) = \frac{dF_0}{dx} = \text{pdf of } x \text{ given } D=0$$

Define $TPR$ and $FPR$: $$TPR(x) = 1- F_1(x) = P(X > x | D = 1) = P(X_1 > x)$$ $$FPR(x) = 1 - F_0(x) = P(X > x | D = 0) = P(X_0 > x)$$ $$dFPR(x) = -f_0(x)$$ The ROC curve is $$ROC(\cdot) = \{(FPR(x), TPR(x)), x \in (\infty, -\infty) \}$$ I reversed the $\infty$ and $-\infty$ to remind myself that the ROC curve is traced out in the unit square from the left lower corner $(0, 0)$ to the right upper corner $(1, 1)$ as $x$ decreases from $\infty$ to $-\infty$.

The ROC curve has the following representation: $$ROC(t) = TPR(FPR^{-1}(t)) \quad t = FPR(x) \quad t \in (0,1)$$ The area under the ROC curve is \begin{align*} AUROC = \int_0^1 ROC(t)dt &= \int_0^1 TPR(FPR^{-1}(t))dt \\ &= \int_{FPR^{-1}(0)}^{FPR^{-1}(1)} TPR(x)dFPR(x)\\ &= \int_{\infty}^{-\infty} TPR(x)dFPR(x)\\ &= \int_{\infty}^{-\infty} TPR(x)(-f_0(x))dx\\ &= \int_{-\infty}^{\infty}TPR(x)f_0(x) dx\\ &= \int_{-\infty}^{\infty}(1-F_1(x))f_0(x) dx\\ &= \int_{-\infty}^{\infty}P(X_1>x) f_0(x) dx \end{align*} This last expression leads to the interpretation of the AUROC as the probability that a random individual with the condition ($D=1$) will have a higher value of $X$ than a random individual without the condition ($D=0$).

Continuing from above: \begin{align*} AUROC &= \int_{-\infty}^{\infty}P(X_1 > x)f_0(x) dx \\ &= P(X_1 > X_0) \end{align*} Thus the AUROC is equal to the probability that $X_1$ and $X_0$ from a randomly selected pair of individuals with $D=1$ and $D=0$ will be correctly ordered, $P(X_1 > X_0)$.

I adapted this from Pepe page 78.

Once you accept that $AUROC = P(X_1 > X_0)$, then deriving a formula for the area under the binormal ROC curve is straightforward.

For the binormal ROC curve, $$X_1 \sim N(\mu_1, \sigma_1^2)$$ $$X_0 \sim N(\mu_0, \sigma_0^2)$$ So, \begin{align*} AUROC &= P(X_1 > X_0)\\ &= P(X_1- X_0 > 0)\\ \end{align*} The difference of two independent normal variables has a normal distribution: $$X_1 - X_0 \sim N(\mu_1 - \mu_0, \sigma_1^2+\sigma_0^2)$$ So, \begin{align*} P(X_1- X_0 > 0) &= 1 - \Phi\left(\frac{0 - (\mu_1 -\mu_0)}{\sqrt{\sigma_1^2+\sigma_0^2}}\right)\\ &= \Phi\left(\frac{\mu_1-\mu_0}{\sqrt{\sigma_1^2+\sigma_0^2}} \right) \end{align*}