I am having trouble understanding the following steps in the Bayesian approach to predicting the probability of tossing a head H given some data D

\begin{aligned} p(H \mid D) & =\int_0^1 p(H, w \mid D) d w \\ & =\int_0^1 p(H \mid w, D) p(w \mid D) d w \end{aligned}

I understand the first step is the sum rule and the second is the chain rule for probability.

I am thinking I should be looking at the integrand as $P((H\cap w) \ \mid D)$ but am having trouble visualising this.

What would be the probability of heads?


Here is the slide from my lecture hand out on Bayesian approach to prediction. Probability of head given data

There is also information on Wikipedia


1 Answer 1


Do you have a link to somewhere that claims this is Bayes rule?

As far as I can see line (1) Is marginalization, and line (2) is chain rule. These are all very general probabilistic statements that exists independent of Bayes rule.

So part of your confusion might be that you are trying to look for something where there isn't anything to be found. i.e. there is nothing "inherently Bayesian" in what you have written, as it can exist independently of Bayes rule, as per conventional probability axioms.

What you have done really here is introduce a latent variable $w$ around which some other Bayesian-driven analysis can occur, as there are many latent variable "Bayesian methods" (but what you have written is not Bayes rule, and it requires more construction to be used for Bayesian methods).

Alternatively you could argue in some sense that you have introduced a continuous mixture parameter via $w$. See here.

Also maybe you're thinking of this expression as per the model evidence for a continuous usage of Bayes rule:

$$p(H | D) = \frac{p(D | H) P (H)}{P(D)} = \frac{p(D | H) P (H)}{\int P(D|H)p(H)dH}$$

EDIT: What they are doing is finding a way to "force" in a beta prior distribution by introducing $w$. Normally we dont know $P(H|D)$ at all. We can plot a histogram sure, but what equation do we give it? Usually a good prior distribution for coin tosses is the Beta distribution, and this is where the $a$ and $b$ terms come in. So think of $w$ as applying a weighting factor for "How many heads do we expect and how many tails we expect". Naturally as we observe more and more empirical data, we expect the data to give the best answer, and so $|H|$ and $|T|$ should end up dominating the calculations (since $a$ and $b$ are constants so they wont change).

And I think from memory they are applying some variation of the rule of succession to apply the updates where $a=b=1$ and $n=|H|+|T|$, and so $s=|H|$.

  • 1
    $\begingroup$ Thank you. I updated the question. Sorry it took so long. $\endgroup$
    – Kirsten
    Jun 3 at 10:49
  • $\begingroup$ No problem I updated it. If you like the answer please send me an upvote and accept the answer :) I need to farm upvotes atm $\endgroup$ Jun 7 at 1:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.