# Finding the conditional distribution of 2 dependent normal random variables

Here's the situation

$X \sim N(\mu, \sigma^2)$ and given $X=x$, $Y \sim N(x, \tau^2)$

I need to find the distribution of $X$ given $Y=y$

From what's given, I know the pdf's of $X$ as well as $Y|X=x$. By multiplying those pdf's together, I can find the joint distribution of $X$ and $Y$ ($f_{X,Y}=f_Xf_{Y|X=x}$).

This seems useful, but I have no clue how to go about finding $f_Y$ to finish it off (since $f_{X|Y=y} = f_{X,Y}/f_Y$). The integral from $-\infty$ to $y$ of $f_{X,Y}$ with respect to $X$ doesn't have a closed form solution to my knowledge.

Is this possible? Am I on the right track? Any help would be tremendously appreciated!

• After finding the joint distribution, rename "$X$" as "$Y$" and "$Y$" as "$X$", then reverse your steps. – whuber Jan 28 '16 at 20:14
• I'm sorry, but I don't follow you. Do you mean divide the joint pdf by the conditional distribution of X|Y to find the pdf of Y? That defeats the purpose as the conditional distribution of X|Y is ultimately what I'm looking for. – CoolBeanz Jan 28 '16 at 21:12
• Why do you think you have to take the integral from $0$ to $y$ to get the marginal distribution? The normal distribution is defined over the real line. – JohnK Jan 28 '16 at 21:59
• Oops. Typo. My bad. Fixed it in the edit. I don't think it makes the integral any more "computable" though... – CoolBeanz Jan 28 '16 at 23:00
• Also asked on math.SE. Please don't crosspost – Dilip Sarwate Jan 29 '16 at 4:55

Suppose that $X \sim \mathcal N(\mu,\sigma^2)$ is the value of a signal that we wish to observe, but what we can observe is $X+N$ where $N \sim \mathcal N(0,\tau^2)$ is noise that is independent of $X$. Let $Y$ denote $X+N$. Since $X$ and $N$ are independent, it is straightforward to determine that $Y \sim \mathcal N(\mu, \sigma^2+\tau^2)$. Now, the conditional density of $Y$ given that $X$ has taken on value $x$ is the density of $x+N$ which is clearly $\mathcal N(x,\tau^2)$. What we are asked for, however, is the conditional density of $X$ given that $Y = y$, that is, the a posteriori distribution of $X$ given that we have observed that $Y = y$. We choke down the gorge that is rising in our throats at this blatant Bayesian heresy and note that $X$ and $Y = X+N$ are jointly normal random variables, $X \sim \mathcal N(\mu,\sigma^2)$ and $Y \sim \mathcal N(\mu, \sigma^2+\tau^2)$, and that their covariance is $$\operatorname{cov}(X,Y) = \operatorname{cov}(X,X+N) = \operatorname{cov}(X,X)+ \operatorname{cov}(X,N) = \sigma^2.$$ Now, it is a standard result that when $X$ and $Y$ are jointly normal random variables, the conditional density of $X$ given that $Y = y$ is a normal density with mean
\begin{align}E[X\mid Y = y] &= \mu_X + \left.\left. \frac{\operatorname{cov}(X,Y)}{\operatorname{var}(Y)}\right(y-\mu_Y\right)\\ &= \mu+\left.\left.\frac{\sigma^2}{\sigma^2+\tau^2}\right(y-\mu \right)\\ &= \frac{\sigma^2}{\sigma^2+\tau^2}y +\frac{\tau^2}{\sigma^2+\tau^2}\mu,\end{align} that is, a weighted blend of the observation $y$ and the a priori mean of $X$. Note that if $\sigma^2 \gg \tau^2$, we give large weight to the observation $y$ and very little to the known mean $\mu$, while if $\sigma^2 \ll \tau^2$, we tend to ignore the observation and give heavy weight to the mean $\mu$. Also, the variance of this conditional density is $$\operatorname{var}(X)\left(1-\frac{(\operatorname{cov}(X,Y))^2}{\operatorname{var}(X)\operatorname{var}(Y)}\right) = \sigma^2\left(1-\frac{\sigma^4}{\sigma^2(\sigma^2+\tau^2)}\right)= \frac{\sigma^2\tau^2}{\sigma^2+\tau^2}.$$ This allows us to write down the conditional density $f_{X\mid Y}(x\mid y)$ directly without messing up the ranges of integrations etc. as in the OP's approach.
• But $f_{X,Y}$ isn't the pdf of a multivariate normal distribution. Or am I wrong about that? – CoolBeanz Jan 28 '16 at 23:06