2
$\begingroup$

I found two expressions for a Bayesian risk classifier when the posteriori probability is unknown, but I don't understand how and why the derivations were made.

For this scenario, assume:

$X\in\mathbb{X}=[0,1],Y\in\{ 0,1 \}$

$\pi_y=P(Y=y)=1/2$ for $y\in{0,1}$

Conditional distributions $[X|Y=y]$ characterised by:

$f(x|Y=0)=2-2x$ and $f(x|Y=1)=2x$.

Let $\tau_1$ be the posteriori probability and $L(r*)$ be the risk classifier.

In the first case, assume $\tau_1\in[0,1]$ is unknown, thus the following expression can be written: $L(r^*)=\int_Xmin\{(1-\tau_1)f(x|Y=0),\tau_1f(x|Y=1\}dx$

I'd like to understand this expression and get an intuitive understanding why it is true?

Additionally, if $\tau_0=\tau_1=1/2$, the following expression can be derived: $L(r*)=1/2-1/4\int_X|f(x|Y=1)-f(x|Y=0)|dx$

How is this connected to the above statement and why is it true?

$\endgroup$
3
  • $\begingroup$ lease explain (a) what does $\tau_1$ stand for? Is it the conditional probability $P(Y=1|X)$? (b) is this question related to your previous one? (stats.stackexchange.com/questions/549134/…) $\endgroup$
    – Spätzle
    Oct 24 at 8:30
  • $\begingroup$ For $(a)$, $\tau_1$ is the posteriori probability. I've seen it referred to as $\mu$ in other literature. For $(b)$, yes it is connected to that question. This is from a series of questions on the same conditions and data. $\endgroup$ Oct 24 at 8:45
  • 1
    $\begingroup$ please add the full details of the problems, as other readers do not remember them by heart $\endgroup$
    – Spätzle
    Oct 24 at 10:44
0
$\begingroup$

We have previously seen that $L(r^*)$ is $$L(r^*)=E\left[\min\{\tau_1(X),1-\tau_1(X)\}\right]=\frac{1}{2}-\frac{1}{2}E\left[\left|2\tau_1(X)-1\right|\right]$$

The loss function is the probability of producing a wrong prediction, given $X=x$:

$$L(r^*)=P(r^*(x)\ne Y|X)=P(\tau_1<1-\tau_1,Y=1|X)+P(\tau_1>1-\tau_1,Y=0|X)$$ We use the property of indicator function for event $A$, which is $P(A)=E[I\{A\}]$: $$=E[I\{\tau_1<1-\tau_1,Y=1|X\}]+E[I\{\tau_1>1-\tau_1,Y=0|X\}]$$ Next thing we need to know is that and indicator of two events can be decomposed to a multiplication of two indicators (like an AND function):

$$=E[I\{\tau_1<1-\tau_1|X\}]\cdot E[I\{Y=1|X\}]+E[I\{\tau_1>1-\tau_1|X\}]\cdot E[I\{Y=0|X\}]$$

Again, indicator property:

$$=E[I\{\tau_1<1-\tau_1|X\}]\cdot P(Y=1|X)+E[I\{\tau_1>1-\tau_1|X\}]\cdot P(Y=0|X)$$

and using Bayes' law:

$$L(r^*)=E[I\{\tau_1<1-\tau_1|X\}]\cdot 0.5P(X|Y=1)+E[I\{\tau_1>1-\tau_1|X\}]\cdot 0.5P(X|Y=0)\\ 0.5E[I\{\tau_1<1-\tau_1|X\}]\cdot f(x|Y=1)+0.5E[I\{\tau_1>1-\tau_1|X\}]\cdot f(x|Y=0)$$

Now, let's decompose $E[I\{\tau_1<1-\tau_1|X\}]$: $\tau_1$ is already a function of $x$, so the conditional notation is redundant, meaning $E[I\{\tau_1<1-\tau_1|X\}]=E[I\{\tau_1<1-\tau_1\}]$. We apply the law of total expectation: $E[I\{\tau_1<1-\tau_1\}]=E[E[I\{\tau_1<1-\tau_1|Y\}]]$. Thing is, $\tau_1$ has nothing to do with $Y$ so:

$$E[E[I\{\tau_1<1-\tau_1|Y\}]]=E[I\{\tau_1<1-\tau_1|Y=0\}+I\{\tau_1<1-\tau_1|Y=1\}]=E[I\{\tau_1<1-\tau_1\}+I\{\tau_1<1-\tau_1\}]=2E[I\{\tau_1<1-\tau_1\}]$$

and a similar result is obtained for $E[I\{\tau_1>1-\tau_1|X\}]$. Plugging these back in, we get (using indicator property):

$$L(r^*)=P(\tau_1<1-\tau_1)f(x|Y=1)+P(\tau_1>1-\tau_1)f(x|Y=0)$$

which can be written as (similar to our previous solution):

$$L(r^*)=E[\min\{\tau_1 f(x|Y=1),(1-\tau_1)f(x|Y=0)\}]$$

and then we simply write the expectation as an integral:

$$L(r^*)=\int_{X}{{\min\{(1-\tau_1)f(x|Y=0),\tau_1f(x|Y=1)\}}dx}\qquad\blacksquare$$


Second part:

$$E[\min\{\tau_1 f(x|Y=1),(1-\tau_1)f(x|Y=0)\}]=\frac{1}{2}E[\min\{f(x|Y=1),f(x|Y=0)\}]$$

Now, the $\min$ function can be written as (check it!):

$$\min\{a,b\}=0.5(a+b-|a-b|)$$

So

$$\frac{1}{2}E[\min\{f(x|Y=1),f(x|Y=0)\}]\\=\frac{1}{4}E[f(x|Y=1)+f(x|Y=0)-|f(x|Y=1)-f(x|Y=0)|]\\=\frac{1}{4}E[2-|f(x|Y=1)-f(x|Y=0)|]\\=\frac{1}{2}-\frac{1}{4}E[|f(x|Y=1)-f(x|Y=0)|]\\=\frac{1}{2}-\frac{1}{4}\int_{X}{|f(x|Y=1)-f(x|Y=0)|dx} $$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.