Intuition for chain rule in Bayesian approach for prediction?

Question

I am having trouble understanding the following steps in the Bayesian approach to predicting the probability of tossing a head H given some data D

\begin{aligned} p(H \mid D) & =\int_0^1 p(H, w \mid D) d w \\ & =\int_0^1 p(H \mid w, D) p(w \mid D) d w \end{aligned}

I understand the first step is the sum rule and the second is the chain rule for probability.

I am thinking I should be looking at the integrand as $P((H\cap w) \ \mid D)$ but am having trouble visualising this.

What would be the probability of heads?

[Update]

Here is the slide from my lecture hand out on Bayesian approach to prediction.

There is also information on Wikipedia

Answer 1

Do you have a link to somewhere that claims this is Bayes rule?

As far as I can see line (1) Is marginalization, and line (2) is chain rule. These are all very general probabilistic statements that exists independent of Bayes rule.

So part of your confusion might be that you are trying to look for something where there isn't anything to be found. i.e. there is nothing "inherently Bayesian" in what you have written, as it can exist independently of Bayes rule, as per conventional probability axioms.

What you have done really here is introduce a latent variable $w$ around which some other Bayesian-driven analysis can occur, as there are many latent variable "Bayesian methods" (but what you have written is not Bayes rule, and it requires more construction to be used for Bayesian methods).

Alternatively you could argue in some sense that you have introduced a continuous mixture parameter via $w$. See here.

Also maybe you're thinking of this expression as per the model evidence for a continuous usage of Bayes rule:

$$p(H | D) = \frac{p(D | H) P (H)}{P(D)} = \frac{p(D | H) P (H)}{\int P(D|H)p(H)dH}$$

EDIT: What they are doing is finding a way to "force" in a beta prior distribution by introducing $w$. Normally we dont know $P(H|D)$ at all. We can plot a histogram sure, but what equation do we give it? Usually a good prior distribution for coin tosses is the Beta distribution, and this is where the $a$ and $b$ terms come in. So think of $w$ as applying a weighting factor for "How many heads do we expect and how many tails we expect". Naturally as we observe more and more empirical data, we expect the data to give the best answer, and so $|H|$ and $|T|$ should end up dominating the calculations (since $a$ and $b$ are constants so they wont change).

And I think from memory they are applying some variation of the rule of succession to apply the updates where $a=b=1$ and $n=|H|+|T|$, and so $s=|H|$.