0
$\begingroup$

I have a book on applied statistics that uses a result I don't understand. The example in the book begins with Bayes' theorem applied to hypothesis $H$ and data $D$:

$$P(H|D) = \frac{P(D|H) P(H)}{P(D|\neg H) P(\neg H) + P(D|H) P(H)} \tag{1}$$

In the example it is given that $P(H) = P(\neg H) = 0.5$. This simplifies the equation for $P(H|D)$ since $P(H)$ and $P(\neg H)$ cancel:

$$P(H|D) = \frac{P(D|H)}{P(D|\neg H) + P(D|H)} \tag{2}$$

It is also given that we do not have a model for $P(D|H)$ so we are to assume that each value of $P(D|H)$ is equally likely (i.e., "the unknown value for $P(D|H)$ is uniformly distributed over the interval $[0,1]$").

Given the uniform distribution of $P(D|H)$, the following equation for $P(H|D)$ is stated:

$$P(H|D) = 1-P(D|\neg H) * ln(\frac{1 + P(D|\neg H)}{P(D|\neg H)}) \tag{3}$$

In the example, $P(D|\neg H)$ is known so equation $(3)$ can be used to solve for $P(H|D)$). This concludes the book's example.

However, I do not understand the justification for going from equation $(2)$ to $(3)$. Can anyone enlighten me as to how assuming $P(D|H)$ is uniformly distributed over $[0,1]$ allows for result $(3)$?

$\endgroup$
  • $\begingroup$ Welcome to our site! Thanks for taking the time to typeset the Latex. On this site we do like short titles, but still try to keep them easy to read (and, especially, easy to search) so I have unabbreviated the title for you. $\endgroup$ – Silverfish Apr 1 '16 at 16:03
2
$\begingroup$

I figured it out.

Since $P(D|H)$ is uniformly distributed over $[0,1]$, you can calculate $P(H|D)$ by taking the integral from $0$ to $1$:

(for ease of reading, I substitute $x = P(D|\neg H)$ and $y = P(D|H)$)

\begin{align} P(H|D) & = \int_0^1 \frac{y}{x+y} dy \\ & = [y - x \cdot \ln(y+x)] \Big|_{y=0}^{y=1} \\ & = 1 - x \cdot \ln(1 + x) + x \cdot \ln x \\ & = 1 - x \cdot \big( \ln(1+x) - \ln x \big) \\ & = 1 - x \cdot \ln \frac{1+x}{x} \end{align}

Replace $x = P(D|\neg H)$ and you have result $(3)$ in my question above:

$$P(H|D) = 1 - P(D|\neg H) \cdot \ln(\frac{1+P(D|\neg H)}{P(D|\neg H)})$$

I hope this helps someone else.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.