# Posterior probability distribution after observing two heads

Say I have a coin and I don't know what is the probability of getting heads. So I set $$\ p(H) = \theta$$ where $$\ \theta \sim U(0,1)$$

Suppose I have flipped the coin once and observed one head so according to this formula:

$$\ (1)\ \ \ f( \theta | y) = \frac{f(y|\theta)\cdot f(\theta)}{\int_{-\infty}^{\infty} f(y | \theta ) \cdot f(\theta) \ d\theta} = \frac{likelihood \cdot prior}{normalizing \ constant}$$

the probability of heads (as shown by the teacher) will be

$$\ (2) \ \ \ f(\theta \ | \ y = 1) = \frac{\theta^1(1-\theta)^{1-y}}{\int_0^1\theta^1(1-\theta)^{1-y}d\theta} = \frac{\theta}{\int_0^1 \theta d\theta} = \frac{\theta}{1/2} = 2\theta$$

now suppose I have flipped again and observed another head, what will be the posterior probability of getting head?

If I understand correctly then $$\ (y_1 | \theta) \sim Bernulli(\theta)$$ and I flipped the coin more than once then $$\ (y | \theta) \sim Binomial(n, \theta)$$ but what exactly is "normalizing constant" and why did the teacher just dropped $$\ f(\theta)$$ from his calculations in $$\ (2)$$ ?

The prior here is uniformly distributed over $$[0,1]$$. Therefore the probability density function (pdf) is $$f(\theta)=1$$ for $$\theta\in[0,1]$$, and $$f(\theta)=0$$ for $$\theta\not\in[0,1]$$. So the integral over $$\mathbb R$$ turned into the integral over $$[0,1]$$ where $$f(\theta)=1$$.
For two coin flips the probability that we observe two heads $$y_1=1,y_2=1$$ under fixed value of $$\theta$$ is $$f(y_1=1,y_2=1\mid \theta)=\mathbb P(\text{two heads}\mid \theta)=\theta^2$$ and normalizing constant is $$\int_{-\infty}^{\infty} f(y_1=1,y_2=1 | \theta ) \cdot f(\theta) \ d\theta = \int_0^1\theta^2\cdot 1\,d\theta = \frac13.$$ So for $$\theta\in[0,1]$$, pdf of posterior distribution is $$f(\theta \mid y_1=1,y_2=1) = \frac{\theta^2}{1/3}=3\theta^2.$$
• Thanks for your answer. Can you please explain what is normalizing constant? Also, does it mean that $\ f(\theta)$ is just an indicator so if $\ \theta \in [0,1]$ it is $\ 1$ and else it's $\ 0$ ? Why do we need it? It's not like $\ \theta$ could be anything else than $\ [0,1]$ ?? Jan 1 '20 at 15:32
• The function $f(\theta|y)=\theta^2$ is not a valid density on $[0,1]$ since it does not integrates to $1$. Therefore normalizing constant $c$ is needed s.t. $f(\theta|y)=\frac{\theta^2}{c}$ integrates to 1. And this constant is exactly $\int_0^1 \theta^2 d\theta=1/3$. With this constant $f(\theta|y)=3\theta^2$ is the valid density: $\int_0^1 3\theta^2 d\theta=1$. Yes, density of uniform distribution is $\mathbb 1_{[0,1]}$.