1
$\begingroup$

For fun I was reading some notes on Operating Characteristics and it said (paraphrased with notation definition):

As Fig. 1 suggests (omitted from question), good detection probability $P_D = P(\hat{H}(y) = H_1 \mid H = H_1)$ is generally obtained at the expense of high false alarm probability $P_F = P(\hat{H}(y) = H_1 \mid H = H_0)$ and so choosing a (Likely Ratio Test LRT) threshold $\eta$ for a particular problem involves making an acceptable tradeoff... From this perspective, the Bayesian Hypothesis test (i.e. choosing a decision rule that minimizing Bayes risk $\varphi(f) = \mathbb{E}_{H,y}[C(f(y), H)] $) represents a particular tradeoff, and corresponds to a single point on this curve. To obtain this tradeoff, we effectively selected as our objective function a linear combination of $P_D$ and $P_F$. More specifically, we minimize: $$ \varphi(f) = \alpha P_F - \beta P_D + \gamma$$ over all possible decision rules, where the choice of $\alpha$ and $\beta$ is in turn, determined by the cost assignment $C_{ij}$'s (where i corresponds to the chosen hypothesis by the decision rule and j corresponds to the correct hypothesis) and the a priori probabilities $P_m$'s.

and claims for the Binary case that the following is true:

$$\alpha = (C_{10} - C_{00})P_0,$$ $$\beta=(C_{01} - C_{11})P_1,$$ $$ \gamma = (C_{00} + C_{01})P_1$$

My first question is, why do we minimize:

$$ \varphi(f) = \alpha P_F - \beta P_D + \gamma$$

that doesn't quite makes sense to me. Why do we want to minimize that? Is it because we just want to minimize the expected risk? (Also, why does that equation express the intrinsic trade off between $P_D$ and $P_F$?)From the notation, I infer that one can re-write Bayes Risk and write it in that form, however, I've been unable to do that. Here is what I have tried:

$$ \varphi(f) = \mathbb{E}_{H,y}[C(f(y), H)] = \sum_{H_j,y} C(f(y), H_j) P(y,H_j)$$

$$ \varphi(f) = \sum_{H_j} \sum_{H_i} \sum_{y : f(y) = H_i} C(f(y) = H_i, H_j) P(y \mid H_j) P(H_j)$$ $$ =\sum_{H_j,H_i} C(H_i, H_j) P(f(y) = H_i \mid H_j) P(H_j)$$ for the binary case we have (were $P_i = P(H_i)$, $C_{ij} = C(f(y) = H_i, H_j)$, $P(H_i \mid H_j) = P(f(y) = H_i \mid H = H_j)$):

$$ \varphi(f) = C_{00} P(H_0 \mid H_0)P_0 + C_{11} P(H_1 \mid H_1)P_1 + C_{10} P(H_1 \mid H_0)P_0 + C_{01} P(H_0 \mid H_1)P_1$$

$$ \varphi(f) = C_{00} P(H_0 \mid H_0)P_0 + C_{11} P_DP_1 + C_{10} P_F P_0 + C_{01} P(H_0 \mid H_1)P_1$$

and after this step, is where I get stuck. The reason I am having trouble advance is because, the form the text suggests as an answer does not even have the terms $P(f(y) = H_0 \mid H = H_0) = P(H_0 \mid H_0)$ and $ P(f(y) = H_0 \mid H = H_1) = P(H_0 \mid H_1)$ anywhere. Which makes me suspect that there might be a typo on the text and $\gamma$ should actually be something else involving $P(H_0 \mid H_1)$ or $P(f(y) = H_0 \mid H = H_0)$. Even if that were true, I can't explain myself how Bayes Risk has a minus sign (as in $\alpha P_F - \beta P_D + \gamma$) even if there were true, unless it was cancelled out somewhere again in the $\gamma$ by some hidden terms. Someone know where the typo is or if there is a way of proceeding from the location I got stuck?

$\endgroup$
1
  • $\begingroup$ realized what the answer is, using $1 - P_D$ and $1 - P_F$, its easy to simplify and find what the constant terms should be $\alpha , \beta, \gamma$. $\endgroup$ Commented Dec 31, 2015 at 6:09

1 Answer 1

0
$\begingroup$

The goal of the question is to show that in the (Binary) Bayesian Hypothesis Testing framework, one can write the expected risk $\varphi(f) = \mathbb{E}_{H,y}[C(f(y), H)]$ as follow:

$$ \varphi(f) = \alpha P_F - \beta P_D + \gamma$$

were $\alpha, \beta, \gamma $ are constants that depend on the cost function ($C_{i,j} = C(f(y) = H_i, H = H_j)$) chosen and the prior probabilities on the hypothesis ($P_m$'s).

So lets write down what $ \varphi(f)$ means and do some algebra:

$$ \varphi(f) = \mathbb{E}_{H,y}[C(f(y), H)] = \sum_{H_j,y} C(f(y), H_j) P(y,H_j)$$

$$ \varphi(f) = \sum_{H_j} \sum_{H_i} \sum_{y : f(y) = H_i} C(f(y) = H_i, H_j) P(y \mid H_j) P(H_j)$$ $$ \varphi(f) =\sum_{H_j,H_i} C(H_i, H_j) P(f(y) = H_i \mid H_j) P(H_j)$$ for the binary case we have (were $P_i = P(H_i)$, $C_{ij} = C(f(y) = H_i, H_j)$, $P(H_i \mid H_j) = P(f(y) = H_i \mid H = H_j)$):

$$ \varphi(f) = C_{00} P(H_0 \mid H_0)P_0 + C_{11} P(H_1 \mid H_1)P_1 + C_{10} P(H_1 \mid H_0)P_0 + C_{01} P(H_0 \mid H_1)P_1$$

$$ \varphi(f) = C_{00} P(H_0 \mid H_0)P_0 + C_{11} P_DP_1 + C_{10} P_F P_0 + C_{01} P(H_0 \mid H_1)P_1$$

Now notice that we want to express the expected risk in terms of the detection probability and false alarm probability only. For this we need to find expression for $P(H_0 \mid H_0)$ and $P(H_0 \mid H_1)$ in terms of $P_F$ and $P_D$. Its easy to see that the equations that we need are:

$$P(H_0 \mid H_0) = 1 - P_F$$ and $$ P(H_0 \mid H_1)$ = 1 - P_D$$

So the expected risk $ \varphi(f)$ becomes:

$$ \varphi(f) = C_{00} (1 - P_F) P_0 + C_{11} P_D P_1 + C_{10} P_F P_0 + C_{01} (1 - P_D) P_1$$

$$ \varphi(f) = (C_{10} - C_{00}) P_0 P_F - (C_{01} - C_{11})P_1 P_D + C_{00}P_0 + C_{01} P_1$$

Thus:

$$\alpha = (C_{10} - C_{00}) P_0 $$ $$\beta = (C_{01} - C_{11})P_1 $$ $$\gamma = C_{00}P_0 + C_{01} P_1 $$

as required $ \varphi(f) = \alpha P_F - \beta P_D + \gamma$.

To finish with some interesting remarks, why would we want to express the expected risk like that? Well, the interesting thing is that if one plots an operating characteristic (OC) for some hypothesis test, one can see how a choice of priors determines which point $(P_D, P_F)$ on the OC we will have. So a choice of priors and costs intrinsically chooses a point $(P_D, P_F)$.

Also, notice that there is a constant $\gamma$ that is irreducible (i.e. independent of which decision rule $f$ we choose). Thus, no matter what we do (unless we choose costs and priors that are exactly zero as to make $\gamma$ is zero), there might be some irreducible expected error that we will have.

Lastly, if we are able to show there is an intrinsic trade-off between large $P_D$ and small $P_F$, then this equations express this trade off for the bayesian hypothesis testing case. To intuitively see this notice that if one can show that $P_F$ and $P_D$ are positively related (which one can show $\frac{d P_D}{d P_F} \geq 0$) then if $P_D$ is increases (so to decrease expected/bayes risk) then $P_F$ increases too. Thus, its unclear how increasing $P_D$ would necessarily lead to a decrease in the average risk.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.