If both prior and likelihood are Gaussian what can we say about the posterior? If X is a random variable that has Gaussian prior and Gaussian likelihood. What can be inferred about the posterior?
As posterior is proporional to prior*likelihood which are Gaussians, the posterior should also be Gaussians. But I have struggles deriving it.
 A: The exponents in the prior density and the likelihood are added to each other
$$
\frac{(\mu-\mu_0)^2}{\tau^2} + \frac{(\overline x - \mu)^2}{\sigma^2/n}
\quad = \quad \frac{\sigma^2(\mu-\mu_0)^2 + \tau^2(\overline x - \mu)^2}{\sigma^2\tau^2/n} \tag 1
$$
Now let's work on the numerator:
$$
((\sigma^2/n)+\tau^2) \Big(\mu^2 - 2(\mu_0\sigma^2 + \overline x \tau^2)\mu + \text{“constant''} \Big) \tag 2
$$
where “constant” means not depending on $\mu$.
Now complete the square:
$$
\Big(\mu - (\mu_0 \sigma^2 + \overline x \tau^2)\Big)^2 + \text{“constant''}
$$
(where this “constant” will differ from the earlier one, but it just becomes part of the normalizing constant).
So the posterior density is $$ \text{constant} \times \exp\Big( 
\text{negative constant} \times (\mu-\text{something})^2 \Big).$$
In other words, the posterior is Gaussian.
The posterior mean is a weighted average of the prior mean $\mu_0$ and the sample mean $\overline x,$ with weights proportional to the reciprocals variances $\tau^2$ (for the prior) and $\sigma^2/n$ (for the sample mean).
