# Question about deriving posterior distribution from normal prior and likelihood

I am trying to understand how to derive the posterior distribution of a parameter $$\mu$$ given data vector $$z$$, $$P(\mu|z)$$, where $$\mu \sim N(0,A)$$ and $$z|\mu \sim N(\mu,1).$$

Obviously from Bayes theorem $$P(\mu|z) = g(\mu) f(z|\mu) /f(z).$$ where $$f(z)= \int g(\mu) f(z|\mu) d \mu .$$

I can write $$g(\mu) f(z|\mu)$$ as a pointwise product of normal densities:

$$\frac{1}{2 \pi \sqrt{A}} \exp \bigg(-\frac{\mu^2+A(z-\mu)^2}{2A}\bigg)$$

the solution is given in Efron's book on Large Scale Inference, p. 2. here:

$$\mu|z \sim N(zB,B)$$ where $$B=\frac{A}{A+1}$$

I would appreciate advice on how to approach the problem (and the answer to this question is a proof). In particular I do not understand what to do with the integral in the numerator of Bayes theorem.

EDIT Following answer by @Neil_G I took a next approach:

We have: \begin{align} g(\mu) &\propto \exp\bigg(- \frac{\mu^2}{2A}\bigg) \\[8pt] f(z|\mu) &\propto \exp\bigg(z \mu - \frac{1}{2}(z^2+\mu^2)\bigg) \end{align}

so \begin{align} P(\mu|z) &\propto \exp\bigg(- \frac{1}{2A} \mu^2 + z\mu - \frac{1}{2} z^2 - \frac{1}{2} \mu^2\bigg) \\[8pt] &= \exp \bigg(-\frac{1}{2B} (\mu^2+Bz^2-2Bz \mu)\bigg) \\[8pt] &\propto \exp \bigg(-\frac{1}{2B} (\mu^2-2Bz \mu)\bigg) \\[8pt] &\propto \exp \bigg(-\frac{ (\mu - Bz)^2}{2B}\bigg) \end{align}

which completes the proof.

• This will be much easier if you write $\mu$ in the the natural parametrization (mean times precision and precision) so that your updates to it are addition. – Neil G Dec 6 '15 at 17:54
• Also forget about $f(z)$ — just write Bayes rule with $\propto$ – Neil G Dec 6 '15 at 17:55
• What do you mean by natural parameterization? – tomka Dec 6 '15 at 17:56
• it's the parametrization $a, b$ such that a normal density can be written $f(x \mid a, b) \propto \exp(ax + bx^2)$. Therefore, the pointwise product of densities having parameters $a_1, b_1$ and $a_2, b_2$ is simply the sum $(a_1+ a_2, b_1+b_2)$. – Neil G Dec 6 '15 at 17:58
• Complete the square w.r.t $\mu$ to find the solution. – hard2fathom Dec 6 '15 at 18:04

First note that \begin{align} x \sim N(\mu, \sigma^2) &\Leftrightarrow f(x \mid \mu, \sigma^2) \propto \exp\left(\frac\mu{\sigma^2}x -\frac1{2\sigma^2}x^2\right) & \tag{1} \\ x \sim N(\mu, \sigma^2) &\Rightarrow L(\mu \mid x, \sigma^2) \propto \exp\left(\frac{x}{\sigma^2}\mu -\frac1{2\sigma^2}\mu^2\right) & \tag{2} \end{align}

So, \begin{align} P(\mu) &\propto \exp\left(-\frac{1}{2A}\mu^2\right) & \text{by (1)} \\ L(\mu \mid z) &\propto \exp\left(\mu z - \frac12\mu^2\right) & \text{by (2)} \\ \implies P(\mu \mid z) &\propto P(\mu) L(\mu \mid z) \\ &\propto \exp\left(\mu z - \frac{A + 1}{2A}\mu^2\right) \\ \implies \mu &\sim N\left(z\frac{A}{A+1}, \frac{A}{A+1}\right) & \text{by (1)}. \end{align}

• (Not sure if I made any mistakes, but this is how I would go about it.) – Neil G Dec 6 '15 at 18:33
• in the second line, don't you forget $-1/2 z^2$? – tomka Dec 6 '15 at 18:47
• @ThomasKlausch That's a constant, so we can forget about it. – Neil G Dec 6 '15 at 18:48
• Thanks, can you check the edit to my post? – tomka Dec 6 '15 at 19:17
• @ThomasKlausch: Glad it makes sense :) I think that's it, yeah. – Neil G Dec 6 '15 at 19:27

The normal prior for the mean of a normal model represents a Conjugate Prior:

https://en.wikipedia.org/wiki/Conjugate_prior

This implies that the posterior of $\mu$ is also normal with certain parameters. This is a classical exercise in any introductory course in Bayesian statistics. The trick consists of expanding the binomial and then factorising in terms of $\mu$ in order to retrieve the normal kernel. Take a look, for instance, at the following lecture notes

http://www.cs.berkeley.edu/~jordan/courses/260-spring10/lectures/lecture5.pdf