0
$\begingroup$

Here's the situation

$X \sim N(\mu, \sigma^2)$ and given $X=x$, $Y \sim N(x, \tau^2)$

I need to find the distribution of $X$ given $Y=y$

From what's given, I know the pdf's of $X$ as well as $Y|X=x$. By multiplying those pdf's together, I can find the joint distribution of $X$ and $Y$ ($f_{X,Y}=f_Xf_{Y|X=x}$).

This seems useful, but I have no clue how to go about finding $f_Y$ to finish it off (since $f_{X|Y=y} = f_{X,Y}/f_Y$). The integral from $-\infty$ to $y$ of $f_{X,Y}$ with respect to $X$ doesn't have a closed form solution to my knowledge.

Is this possible? Am I on the right track? Any help would be tremendously appreciated!

$\endgroup$
  • 1
    $\begingroup$ After finding the joint distribution, rename "$X$" as "$Y$" and "$Y$" as "$X$", then reverse your steps. $\endgroup$ – whuber Jan 28 '16 at 20:14
  • $\begingroup$ I'm sorry, but I don't follow you. Do you mean divide the joint pdf by the conditional distribution of X|Y to find the pdf of Y? That defeats the purpose as the conditional distribution of X|Y is ultimately what I'm looking for. $\endgroup$ – CoolBeanz Jan 28 '16 at 21:12
  • $\begingroup$ Why do you think you have to take the integral from $0$ to $y$ to get the marginal distribution? The normal distribution is defined over the real line. $\endgroup$ – JohnK Jan 28 '16 at 21:59
  • $\begingroup$ Oops. Typo. My bad. Fixed it in the edit. I don't think it makes the integral any more "computable" though... $\endgroup$ – CoolBeanz Jan 28 '16 at 23:00
  • $\begingroup$ Also asked on math.SE. Please don't crosspost $\endgroup$ – Dilip Sarwate Jan 29 '16 at 4:55
3
$\begingroup$

Adapted from this previous answer of mine.

Suppose that $X \sim \mathcal N(\mu,\sigma^2)$ is the value of a signal that we wish to observe, but what we can observe is $X+N$ where $N \sim \mathcal N(0,\tau^2)$ is noise that is independent of $X$. Let $Y$ denote $X+N$. Since $X$ and $N$ are independent, it is straightforward to determine that $Y \sim \mathcal N(\mu, \sigma^2+\tau^2)$. Now, the conditional density of $Y$ given that $X$ has taken on value $x$ is the density of $x+N$ which is clearly $\mathcal N(x,\tau^2)$. What we are asked for, however, is the conditional density of $X$ given that $Y = y$, that is, the a posteriori distribution of $X$ given that we have observed that $Y = y$. We choke down the gorge that is rising in our throats at this blatant Bayesian heresy and note that $X$ and $Y = X+N$ are jointly normal random variables, $X \sim \mathcal N(\mu,\sigma^2)$ and $Y \sim \mathcal N(\mu, \sigma^2+\tau^2)$, and that their covariance is $$\operatorname{cov}(X,Y) = \operatorname{cov}(X,X+N) = \operatorname{cov}(X,X)+ \operatorname{cov}(X,N) = \sigma^2.$$ Now, it is a standard result that when $X$ and $Y$ are jointly normal random variables, the conditional density of $X$ given that $Y = y$ is a normal density with mean

\begin{align}E[X\mid Y = y] &= \mu_X + \left.\left. \frac{\operatorname{cov}(X,Y)}{\operatorname{var}(Y)}\right(y-\mu_Y\right)\\ &= \mu+\left.\left.\frac{\sigma^2}{\sigma^2+\tau^2}\right(y-\mu \right)\\ &= \frac{\sigma^2}{\sigma^2+\tau^2}y +\frac{\tau^2}{\sigma^2+\tau^2}\mu,\end{align} that is, a weighted blend of the observation $y$ and the a priori mean of $X$. Note that if $\sigma^2 \gg \tau^2$, we give large weight to the observation $y$ and very little to the known mean $\mu$, while if $\sigma^2 \ll \tau^2$, we tend to ignore the observation and give heavy weight to the mean $\mu$. Also, the variance of this conditional density is $$\operatorname{var}(X)\left(1-\frac{(\operatorname{cov}(X,Y))^2}{\operatorname{var}(X)\operatorname{var}(Y)}\right) = \sigma^2\left(1-\frac{\sigma^4}{\sigma^2(\sigma^2+\tau^2)}\right)= \frac{\sigma^2\tau^2}{\sigma^2+\tau^2}.$$ This allows us to write down the conditional density $f_{X\mid Y}(x\mid y)$ directly without messing up the ranges of integrations etc. as in the OP's approach.

$\endgroup$
  • $\begingroup$ You helped me even though I'm a cross posting scum... for this, I am eternally grateful, and I vow to put an end to my cross posting ways. $\endgroup$ – CoolBeanz Feb 23 '16 at 20:56
0
$\begingroup$

Once you know the joint distribution, just use the property of the multivariate normal that the conditional distributions are also normal with parameters as defined here: https://en.wikipedia.org/wiki/Multivariate_normal_distribution#Conditional_distributions. Just fix Y to some value rather than fixing X.

$\endgroup$
  • $\begingroup$ But $f_{X,Y}$ isn't the pdf of a multivariate normal distribution. Or am I wrong about that? $\endgroup$ – CoolBeanz Jan 28 '16 at 23:06
  • $\begingroup$ The normal prior on x is the conjugate prior for the mean of a normally distributed random variable. This means that the posterior (which is proportional to the prior distribution on x times the likelihood of y) will also be normal. If you work it out, you should get this. $\endgroup$ – Vivek Subramanian Jan 28 '16 at 23:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.