1
$\begingroup$

The question comes from Kevin Murphy's book, Ch 5, Ex 5.6. Could somebody suggest a solution?

Let $B=p(D|H_1)/p(D|H_0)$ be the bayes factor in favor of model 1. Suppose we plot two ROC curves, one computed by thresholding $B$, and the other computed by thresholding $p(H_1|D)$. Will they be the same or different? Why?

$\endgroup$
0
$\begingroup$

By using Bayes' theorem we can write

$B = \frac{p(D|H_1)}{p(D|H_0)} = \frac{p(H_1|D)p(H_0)}{p(H_0|D)p(H_1)} \propto \frac{p(H_1|D)}{p(H_0|D)}$

From this we see that ROC curve obtained by thresholding $B$ will be the same as ROC curve obtained by thresholding $p(H_1|D)$ only if $p(H_0|D)$ is constant.

$\endgroup$
  • $\begingroup$ You can write the final term as $B \propto p/(1-p)$, so it’s just a monotonic transformation, leaving ROC invariant (assuming only 2 hypotheses) $\endgroup$ – innisfree May 13 '19 at 13:38
0
$\begingroup$

Sinc we are discussing ROC curves, let me assume we are taking about two hypothesis. We can write the Bayes factor as $$ B_{10} = \frac{p_1}{1−p_1} \frac{\pi_0}{\pi_1} $$ where $p_1$ is the posterior of the alternative and $\pi_0/\pi_1$ are the prior odds.

Note well that the relationship between $p_1$ and $B$ is monotonic. The ROC is invariant under such monotonic transformations (as it only depends on the ordering of the discriminant, not it's absolute value). So the ROC curves will be the same.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.