Signal extraction problem: conditional expectation of one item in sum of independent normal RVs

Question

I have what I'm afraid is a simple stats problem that is stumping me. I have two random variables, X and Y, independently normally distributed:

 X ~ N(0, sigmaX)
 Y ~ N(0, sigmaY).

I observe the sum of these two variables, Z = X+Y, and want to develop a conditional expectation on X given the sum. A colleague said, "ah, yes, classic signal-extraction problem. Solution is:"

 E[X|X+Y] = (X + Y) * sigmaX / (sigmaX + sigmaY)

This looked about right so I thanked him and figured I work it out at home. It appears I'm a little rusty here, though. I can give verbal reasoning why this would be true but can't write down the math. What is the mathematical reason this is true?

Thanks all!

Answer 1

If $X$ and $Y$ are zero-mean independent normal random variables with variances $\sigma_X^2$ and $\sigma_Y^2$ respectively, then $X$ and $X+Y = Z$ are zero-mean jointly normal variables where $\sigma_Z^2 = \sigma_X^2 + \sigma_Y^2$ and $\text{cov}(X, Z) = \text{cov}(X,X+Y) = E[X^2] + E[XY] = \sigma_X^2$ so that the correlation coefficient is $$\rho_{X,Z} = \frac{\sigma_X^2}{\sigma_X\sqrt{\sigma_X^2 + \sigma_Y^2}} = \frac{\sigma_X}{\sqrt{\sigma_X^2 + \sigma_Y^2}}$$. The conditional distribution of $X$ given $Z = z$ is normal, and since the variables involved all have zero means, the conditional mean simplifies to
$$z\cdot \rho_{X,Z}\frac{\sigma_X}{\sigma_Z} = z\cdot \frac{\sigma_X^2}{\sigma_X^2 + \sigma_Y^2}$$

Answer 2

It is just like the linear model $Z = X + \epsilon$ (take $\epsilon \equiv Y$), but where you regress $X$ on $Z$ rather than $Z$ on $X$.

The slope of the regression of $X$ on $Z$ is $\text{cor}(X,Z) \sigma_X / \sigma_Z$, and $\sigma^2_Z = \sigma^2_X + \sigma^2_Y$, while $\text{cor}(X,Z) = \text{cov}(X,Z)/[\sigma_X \sigma_Z] = \sigma_X^2/[\sigma_X \sigma_Z] = \sigma_X / \sqrt{\sigma_X^2+\sigma_Y^2}$.

Thus I get the slope to be $\sigma_X^2/(\sigma_X^2+\sigma_Y^2)$

So, the same as you, provided that sigmaX and sigmaY are interpreted as variances.