I have a question about Bayesian updating. In general Bayesian updating refers to the process of getting the posterior from a prior belief distribution.
Alternatively one could understand the term as using the posterior of the first step as prior input for further calculation.
The below is a simple calculation example. Method a is the standard calculation. Method b uses the posterior output as input prior to calculate the next posterior.
Using method a, we get P(F|HH) = 0.2. Using method b, gives P(F|HH) = 0.05. My question is as to how far method b is a valid approach ?
Problem: You toss a coin twice, get 2 Heads. What is the probability that the coin is fair, i.e. $Pr(Fair\ coin| HH)$?
Now for the first toss: $Pr(Fair\ coin| H) = \frac{Pr(Head|Fair)\cdot P(Fair)}{Pr(Head|Fair) \cdot P(Fair)+Pr(Head|Biased) \cdot P(Biased)} = \frac{Pr(H|F)\cdot P(F)}{P(H)} \quad\quad (1)$
Assuming starting prior belief P(Fair) = 0.5, want to find P(F|H) for the first toss
Below are the calculation for the intermediate steps:
$P(H|F)= {n \choose x} \theta^{x}(1-\theta)^{n-x} = {1 \choose 1} 0.5^{1}(0.5)^{0}= 0.5$
$P(H)= P(H|F) \cdot P(F)+ P(H|Biased) \cdot P(Biased)=(0.5 \cdot 0.5) +(1 \cdot 0.5) = 0.75$
(Note: P(H|Biased) = 1 because assuming an extreme example with Heads on both sides of the coin, the probability of getting Heads with a biased coin = 1 (makes calculation easy))
Hence, plugging into (1), we get :
$Pr(F| H) =\frac{Pr(H|F)\cdot P(F)}{P(H)} = \frac{0.5 \cdot 0.5}{0.75} = 0.33$
Now, we toss the coin again and get another H. To calculate $Pr(F| HH) $ , we
a) continue using P(Fair)=0.5
$Pr(F|HH) = \frac{Pr(HH|F)\cdot P(F)}{Pr(HH|F) \cdot P(F)+Pr(HH|Biased) \cdot P(Biased)} = \frac{Pr(HH|F)\cdot P(F)}{P(HH)} \quad\quad (2)$
$P(HH|F)= {n \choose x} \theta^{x}(1-\theta)^{n-x} = {2 \choose 2} 0.5^{2}(0.5)^{0}= 0.25$
$P(HH)= P(HH|F) \cdot P(F)+ P(HH|Biased) \cdot P(Biased)=(0.25 \cdot 0.5) +(1 \cdot 0.5) = 0.625$
Hence, plugging into (2), $Pr(F|HH) =\frac{Pr(HH|F)\cdot P(F)}{P(HH)} = \frac{0.25 \cdot 0.5}{0.625} = 0.2$
Alternatively, what if we calculate $Pr(F| HH) $ by using
b) our updated belief P(Fair)=0.33 which we got from Pr(F|H) in the first step
In this case,
$P(HH|F)= {n \choose x} \theta^{x}(1-\theta)^{n-x} = {2 \choose 2} 0.33^{2}(1-0.33)^{0}= 0.1089$
$P(HH)= P(HH|F) \cdot P(F)+ P(HH|Biased) \cdot P(Biased)=(0.1089 \cdot 0.33) +(1 \cdot 0.67) = 0.705937$
Hence, plugging into (2), $Pr(F|HH) =\frac{Pr(HH|F)\cdot P(F)}{P(HH)} = \frac{0.1089 \cdot 0.33}{0.705937} = 0.05091$
Using method a, we get P(F|HH) = 0.2. Using method b, gives P(F|HH) = 0.05. My question is as to how far method b is a valid approach ?