# Conjugate normal distribution

Given a random variable $X$ which follows a normal distribution $X \sim (\mu,\sigma^2)$ with unknown mean and known variance. Say we have $n$ observations $y_i$ where $1\le i \le n$ and each $y_i$ is sampled from a normal distribution $Y_i \sim N(x,\sigma_i^2)$ (not i.i.d.) where $x$ is sampled from $X$ and it is the same for all $\{Y_i\}$, but $\sigma_i$ is different for different $Y_i$.

The question is: how to compute the posterior distribution $X$ given the observations p(X|{y_i})? The question is different from the answer listed in Wikipedia because $y_i$ is not sampled from i.i.d. distribution.

Here is the setup as I understand it. Let $y = (y_1, \ldots, y_n)$. The sampling distribution for $y$ can be expressed as \begin{equation} p(y|x) = \prod_{i=1}^n \textsf{N}(y_i|x,\sigma_i^2) . \end{equation} Once $y$ is observed, $p(y|x)$ becomes the likelihood for $x$. The prior distribution for $x$ is \begin{equation} p(x) = \textsf{N}(x|\mu,\sigma^2) . \end{equation}
Given this setup, the posterior distribution for $x$ is given by Bayes' rule: \begin{equation} p(x|y) = \frac{p(y|x)\,p(x)}{p(y)} = \textsf{N}(x|m,s^2) , \end{equation} where \begin{align} s^2 &= \left(\frac{1}{\sigma^2} + \sum_{i=1}^n \frac{1}{\sigma_i^2}\right)^{-1} \\ m &= s^2\left(\frac{\mu}{\sigma^2} + \sum_{i=1}^n \frac{y_i}{\sigma_i^2}\right) . \end{align} One may confirm the answer by checking that $p(y|x)\,p(x)/p(x|y)$ does not involve $x$ (after simplification).
The values for $m$ and $s^2$ may be found by using a property of the normal distribution. Given some $K > 0$, let \begin{equation} h(x) = \log\big(K\,\textsf{N}(x|m,s^2)\big) = -\frac{(x-m)^2}{2\,s^2} + C , \end{equation} where $C$ does not involve $x$. We may obtain $m$ by solving $h'(x) = 0$ for $x$. In addition, $-1/h''(x) = s^2$.
We can apply this approach as follows. Let $K = p(y)$. Then $h(x) = \log\big(p(y|x)\,p(x)\big)$.
• Can you give the derivation or a reference of computing $s^2$ and $m$? – Yi Yang Dec 17 '17 at 3:23