0
$\begingroup$

Suppose $X$ is normally distributed with mean $10$ and standard deviation $1$. I take a normally-distributed noisy measurement of $X$ with standard deviation $0.1$, and the measurement is $5$. I am interested in determining the posterior distribution of $X$. Is there a general rule for this?

Thinking about it, I see this is equivalent to: $$X \sim N(10,1) \\ Y \sim N(X,0.1)$$ and then asking $P(X | Y = 5)$, which requires solving: $$P(X=x | Y=5) = \frac{f(x,5,0.1) f(x,10,1)}{ \int_{x'} f(x',5,0.1) f(x',10,1) dx'}$$ where $f$ is the normal density, but that's how far I got.

$\endgroup$
  • $\begingroup$ Standard result. Somewhat messy algebra but conceptually simple application of Bayes' Thm. Result is another normal. Result often simplified by using 'precision' $\tau = 1/\sigma^2$ throughout. Perhaps see these notes. $\endgroup$ – BruceET Dec 25 '18 at 16:28
1
$\begingroup$

For more general situation, let $X\sim N(\mu,\sigma_X^2)$ and noise $\epsilon \sim N(0,\sigma_\epsilon^2)$. Then the measurement $Y=X+\epsilon$.

$$\left( \begin{matrix} X\\Y\end{matrix} \right) = \left( \begin{matrix} 1 &0 \\ 1& 1\end{matrix} \right) \left( \begin{matrix} X \\ \epsilon \end{matrix} \right)$$ Then $\left( \begin{matrix} X\\Y\end{matrix} \right)$ follows the bivariate normal distribution with mean and variance matrix $$E\left( \begin{matrix} X\\Y\end{matrix} \right) = \left( \begin{matrix} \mu\\ \mu\end{matrix} \right)$$

$$Var\left( \begin{matrix} X\\Y\end{matrix} \right) = \left( \begin{matrix} 1 &0 \\ 1& 1\end{matrix} \right) \left( \begin{matrix} \sigma_X^2 &0 \\ 0&\sigma_\epsilon^2\end{matrix} \right) \left( \begin{matrix} 1 & 1 \\ 0& 1\end{matrix} \right) = \left( \begin{matrix} \sigma_X^2 & \sigma_X^2 \\ \sigma_X^2& \sigma_X^2+\sigma_\epsilon^2\end{matrix} \right) $$

The distribution of $(X|Y)$ also follows normal distribution with mean and variance: $$E(X|Y) = \mu +\frac {\sigma_X^2}{\sigma_X^2+\sigma_\epsilon^2}(Y-\mu)$$ $$Var(X|Y) = \sigma_X^2 - \frac {\sigma_X^4}{\sigma_X^2+\sigma_\epsilon^2}$$

$\endgroup$
  • $\begingroup$ Thank you! Could you please indicate the general result used to derive E(X|Y) and Var(X|Y)? It would make the answer even more useful. $\endgroup$ – user118967 Dec 26 '18 at 20:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.