# Deriving posterior from a single observation z from a normal distribution (ESL book)

I am reading the book The Elements of Statistical Learning by Hastie, Tibshirani and Friedman. On page 271 the authors derive a posterior distribution from a single observation $$z\sim N(\theta, 1)$$, where the prior of $$\theta$$ is specified as $$\theta \sim N(0, \tau)$$. It then follows (according to the authors) that the posterior distribution equals $$\theta | z \sim N\left( \frac{z}{1+\frac{1}{\tau}}, \frac{1}{1+\frac{1}{\tau}} \right).$$

Now, my calculations yield

\begin{align} \Pr\left(\theta |\textbf{Z}\right) &= \frac{\Pr\left(\textbf{Z} \mid \theta\right) \Pr(\theta)}{\int \Pr\left(\textbf{Z} \mid \theta\right) \Pr(\theta)d\theta} \propto \Pr\left(\textbf{Z} \mid \theta\right) \Pr(\theta) \\ &= \frac{1}{\sqrt{2\pi}}\exp\left(-\frac{1}{2}(z-\theta)^2 \right) \frac{1}{\sqrt{2\pi\tau}}\exp\left(-\frac{1}{2\tau}\theta^2 \right) \\ &= \frac{1}{2\pi\sqrt{\tau}}\exp\left(-\frac{1}{2} (z^2 + \theta^2 -2z\theta + \frac{\theta ^2}{\tau}) \right) \\ &= \frac{1}{2\pi\sqrt{\tau}}\exp\left(-\frac{1}{2} (\theta^2(1+\frac{1}{\tau}) + z^2 -2z\theta) \right) \\ &= \frac{1}{2\pi\sqrt{\tau}}\exp\left(-\frac{1}{2 \frac{1}{1+\frac{1}{\tau}}} (\theta^2 + \frac{z^2}{1+\frac{1}{\tau}} -2 \frac{z\theta}{1+\frac{1}{\tau}} ) \right). \end{align}

The denominator of $$\frac{z^2}{1+\frac{1}{\tau}}$$ should equal $$(1+\frac{1}{\tau})^2$$ for me to be able to "complete the square" and get

\begin{align} \Pr\left(\theta |\textbf{Z}\right) &\propto \frac{1}{2\pi\sqrt{\tau}}\exp\left(-\frac{1}{2 \frac{1}{1+\frac{1}{\tau}}} (\theta^2 + \frac{z^2}{(1+\frac{1}{\tau})^2} -2 \frac{z\theta}{1+\frac{1}{\tau}} ) \right) \\ &=\text{constant}\times\exp\left(-\frac{1}{2 \frac{1}{1+\frac{1}{\tau}}} (\theta - \frac{z}{1+\frac{1}{\tau}})^2 \right), \end{align}

such that $$\theta | z \sim N\left( \frac{z}{1+\frac{1}{\tau}}, \frac{1}{1+\frac{1}{\tau}} \right)$$.

My question is:

Where do I go wrong in the process? Should I divide with $$\int \Pr\left(\textbf{Z} \mid \theta\right) \Pr(\theta)d\theta = \Pr(\textbf{Z})$$? If so, what is the difference between $$\Pr(\textbf{Z})$$ and $$\Pr\left(\textbf{Z} \mid \theta\right)$$ in this given example?

Best regards,

wanderingashenvalewisp

## 1 Answer

Since we're looking for the pdf of $$\theta$$, we're only concerned with terms that include it.

\begin{align} \Pr\left(\theta |\textbf{Z}\right) &\propto \Pr\left(\textbf{Z} \mid \theta\right) \Pr(\theta) \\ &\propto \exp\left(-\frac{1}{2}(z-\theta)^2 -\frac{1}{2\tau}\theta^2 \right) \\ &= \exp\left(-\frac{1}{2}\left((1+\frac{1}{\tau})\theta^2 -2z\theta+z^2 \right)\right)\\ &= \exp\left(-\frac{1}{2}(1+\frac{1}{\tau})\left(\theta^2 -2\frac{z}{1+\frac{1}{\tau}}\theta+\frac{z^2}{1+\frac{1}{\tau}} \right)\right)\\ &\propto \exp\left(-\frac{1}{2}(1+\frac{1}{\tau})\left(\theta - \frac{z}{1+\frac{1}{\tau}} \right)^2\right) \end{align} And that last line implies the desired result.

• Ah yes, of course, thanks a lot and merry christmas :) Dec 21, 2020 at 17:59