Conditional Gaussian

Question

Suppose I have $$ \begin{align} p(x_1) &= N(x_1; 0, 1) \\ p(x_2 \mid x_1) &= N(x_2; x_1, 1) \end{align} $$ How do I compute $p(x_1 \mid x_2)$? I know how to compute their product, giving $N\left(\frac{x_1}{2}, \frac{1}{2}\right)$. The answer of the exercise is $N\left(\frac{x_2}{2}, \frac{1}{2}\right)$, so with $x_2$ rather than $x_1$ in the mean.

Answer 1

When one Marginal is Normal, and one Conditional is Normal, then the joint bivariate distribution is Normal. Then the other marginal is also Normal, and the second Conditional is also Normal. Using Baye's Law for densities

$$f(x_1 \mid x_2) = \frac{f(x_2 \mid x_1)\cdot f(x_1)}{f(x_2)}$$

we can solve for the left-hand-side. Determining the marginal parameters of $f(x_2)$ as well as the correlation coefficient can be done using the expressions for conditional Normal densities between two variables.

UPDATE responding to OP comments

As mentioned earlier, one marginal and one conditional Normal imply a bivariate Normal distribution, and so both the other conditional and the other Normal will be also Normal. So

$$X_2 \sim N\left(\mu_2, \sigma^2_2\right) , \quad X_1 \mid X_2 \sim N\left (\mu_1+\frac{\sigma_1}{\sigma_2} \rho(x_2 - \mu_2),\,(1-\rho^2)\sigma^2_1\right)$$

and we also have from the premises

$$X_2 \mid X_1 \sim N\left(\mu_2+\frac{\sigma_2}{\sigma_1} \rho(x_1 - \mu_1) =x_1,\,(1-\rho^2)\sigma^2_2 =1\right),$$

$$X_1 \sim N\left(\mu_1=0, \sigma^2_1=1\right) , $$

which gives us

$$\mu_2 = 0, \;\;\; \sigma_2\rho = 1 \implies \sigma_2 = \sqrt{2} , \rho=1/\sqrt{2}.$$

Since

$$X_1 \mid X_2 \sim N\left(\mu_1+\frac{\sigma_1}{\sigma_2} \rho(x_2 - \mu_2) ,\,(1-\rho^2)\sigma^2_1 \right)$$

we get

$$X_1 \mid X_2 \sim N\left(\frac{1}{2}x_2,\,\frac{1}{2}\right),\qquad X_2 \sim N\left(\mu_2 = 0, \sigma^2_2 =2 \right)$$