10
$\begingroup$

Suppose I have two independent normal variables $X$ and $Y$ with known mean and variance. Defining $Z = X+Y$, what is the most straightforward way to compute $\mathbb{E}\left[X|Z\right]$?

I am writing \begin{align} \mathbb{E}\left[X|Z\right] & = \int_x x f_{X}(x|Z=z) \mathrm{d}x \\ & = \int_x x \frac{f_{XZ}(x,z)}{f_Z(z)} \mathrm{d}x \\ & = \int_x x \frac{f_{Y}(z-x)f_X(x)}{f_Z(z)} \mathrm{d}x, \\ \end{align}

but I am not sure if the best (and only?) way is to compute this expression using our knowledge of the pdf's. Thanks.

$\endgroup$
3
  • $\begingroup$ they wouldn't be independent & identically distributed in this case. $\endgroup$
    – qoheleth
    Commented Aug 25, 2014 at 5:39
  • 1
    $\begingroup$ See section 7.4 here: arxiv.org/pdf/1304.7480v2.pdf Hope it was helpful. $\endgroup$
    – sefi
    Commented Aug 25, 2014 at 7:33
  • 1
    $\begingroup$ My answer at stats.stackexchange.com/a/71303 works out the conditional mean and the conditional variance, giving explicit formulas for both (skip to the end at the "Conclusions" section). The conditional mean is the "regression curve." It is given by the standard least-squares regression formulas. Apply this result to the binormal distribution of $(Z,X)$, whose means and covariance matrix are easily computed. (Although it is a lengthy post, the derivation in it is notable for not employing any integrals nor even any explicit representation of the PDF.) $\endgroup$
    – whuber
    Commented Aug 25, 2014 at 13:27

2 Answers 2

3
$\begingroup$

As @StéphaneLaurent points out, $(X,Z)$ have a bivariate normal distribution and $E[X\mid Z] = aZ+b$. But even more can be said in this case because it is known that $$a = \frac{\operatorname{cov}(X,Z)}{\sigma_Z^2}, \quad b = \mu_X - a\mu_Z = \mu_X - \frac{\operatorname{cov}(X,Z)}{\sigma_Z^2}\mu_Z,$$ and we can use the independence of $X$ and $Y$ (which implies $\operatorname{cov}(X,Y) = 0$) to deduce that $$\begin{align} \operatorname{cov}(X,Z) &= \operatorname{cov}(X,X+Y)\\ &= \operatorname{cov}(X,X) + \operatorname{cov}(X,Y)\\ &= \sigma_X^2\\ \sigma_Z^2 &= \operatorname{var}(X+Y)\\ &= \operatorname{var}(X)+\operatorname{var}(Y) + 2\operatorname{cov}(X, Y)\\ &= \sigma_X^2+\sigma_Y^2\\ \mu_Z &= \mu_X+\mu_Y. \end{align}$$ Note that the method used above can also be applied in the more general case when $X$ and $Y$ are correlated jointly normal random variables instead of independent normal random variables.

Continuing with the calculations, we see that $$E[X\mid Z] = \frac{\sigma_X^2}{\sigma_X^2+\sigma_Y^2}(Z-\mu_Z) + \mu_x \tag{1}$$ which I find comforting because we can interchange the roles of $X$ and $Y$ to immediately write down $$E[Y\mid Z] = \frac{\sigma_Y^2}{\sigma_X^2+\sigma_Y^2}(Z-\mu_Z) + \mu_Y\tag{2}$$ and the sum of $(1)$ and $(2)$ gives $E[X\mid Z] + E[Y\mid Z] = Z$ as noted in Stéphane Laurent's answer.

$\endgroup$
6
  • $\begingroup$ There's something strange when comparing our two approaches: mine yields a result that doesn't involve any variance. $\endgroup$ Commented Aug 25, 2014 at 12:25
  • $\begingroup$ @StéphaneLaurent Your approach results in the equations $$\begin{align}\mu_X&=a(\mu_X+\mu_Y)+b\\\mu_Y&=\alpha(\mu_X+\mu_Y)+\beta\\a+\alpha &=1\\b+\beta &=0\end{align}$$ but do these four linear equations give $a$ and $b$ in terms of the means alone? or do they involve $\alpha$ and/or $\beta$ as well? $\endgroup$ Commented Aug 25, 2014 at 12:38
  • $\begingroup$ Thank you both for your answers. @DilipSarwate I recognize your expression for $a$ in terms of covariance/variance as the coefficient in an OLS linear regression. Is that what it is based on or is there a more general idea that both are linked to? $\endgroup$ Commented Aug 25, 2014 at 23:11
  • $\begingroup$ @DilipSarwate After replacing $\alpha$ with $1-a$ and $\beta$ with $-b$, there are two equations and the solutions $a$ and $b$ depends on $\mu_X$ and $\mu_Y$ only. I think my approach is wrong: the system is not invertible. $\endgroup$ Commented Aug 26, 2014 at 10:31
  • 1
    $\begingroup$ @DavidMelkuev Yes, the expressions are based on the essential idea behind OLS: we seek coefficients $a$ and $b$ that minimize the mean-square error $E[(X - (aZ+b))^2]$. This is standard stuff usually covered in undergraduate probability textbooks in the chapters on expectation which usually also point out that for the special case when $X$ and $Z$ have a bivariate normal distribution, the general minimum-mean-square-error (MMSE) estimator $E[X\mid Z]$ coincides with the linear minimum-mean-square error (LMMSE) estimator $aZ+b$. $\endgroup$ Commented Aug 26, 2014 at 11:20
1
$\begingroup$

Each of the pairs $(X,Z)$ and $(Y,Z)$ has a bivariate normal distribution. Then we know that $$E(X\mid Z) = a Z+b \quad\textrm{ and } \quad E(Y \mid Z)=\alpha Z + \beta.$$ Taking the expectation yields $E(X)=aE(Z)+b$ and $E(Y)=\alpha E(Z) + \beta$. But we also have $E(X\mid Z) + E(Y \mid Z) = Z$, therefore $a+\alpha=1$ and $b+\beta=0$. Finally we have to solve a linear system of two equations and two unknown variables.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.