I want to find the covariance between two random variables $X$ and $Y$, which are independent given another random variable $M$. I thought the calculation would be $$cov[X,Y]=E_M[cov[X,Y|M]] =E_M[E_{XY}[(X|M-E_X[X|M])(Y|M-E_Y[Y|M])]].$$ But since $X$ and $Y$ are conditionally independent, $cov[X,Y|M]=0$, and thus, $E_M[cov[X,Y|M]]=0$. Obviously this is false. Would replacing $E_X[X|M]$ with $E_M[E_X[X|M]]$ (and doing the same for $Y$) give me the correct calculation?

A simple yes or no answer will suffice, but I would be especially appreciative of a simple derivation of $cov[X,Y]$ in this situation.


In order to understand why the expression $\operatorname{Cov}(X,Y)=E_M\left[\operatorname{Cov}(X,Y|M)\right]$ is wrong write the definition of conditional covariance

$$\operatorname{Cov}(X,Y\mid M) = E(XY\mid M) - E(X\mid M)E(Y\mid M)$$

Now average over $M$

$$E_M\left[\operatorname{Cov}(X,Y\mid M)\right] = E_M\Big[ E(XY\mid M)\Big] - E_M\Big[ E(X\mid M)\cdot E(Y\mid M)\Big]$$

While $E_M\Big[ E(XY\mid M)\Big] = E(XY)$, the second term cannot be split into $E_M\Big[ E(X\mid M)\Big]\cdot E_M\Big[E(Y\mid M)\Big]$, and so it does not equal $E(X)E(Y)$.

And the reason it cannot be split is that what becomes independent conditional on $M$ are the random variables $\{X,Y\}$ and not the random variables $\big\{ E(X\mid M), E(Y\mid M)\big\}$.

Therefore $\operatorname{Cov}(X,Y)\neq E_M\left[\operatorname{Cov}(X,Y|M)\right]$. The correct expression is

$$\operatorname{Cov}(X,Y)=E_M[\operatorname{Cov}(X,Y \mid M)]+\operatorname{Cov}\big[E(X\mid M),E(Y\mid M)\big]$$

and you can find at Law of Total Covariance, how it is derived.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.