# What is the difference between the sum of two covariance matrices and the covariance matrix of the sum of two variables?

I'm wondering if someone could help to explain the difference between two covariance matrices. Suppose that ${\bf K}_X$ and ${\bf K}_Y$ are two covariance matrices of real random vectors.

What is the difference between ${\bf K}_X+{\bf K}_Y$ and ${\bf K}_{X+Y}$?

-
They are different because ${\bf K}_X + {\bf K}_Y$ is the sum of two covariance matrices while ${\bf K}_{X+Y}$ is the covariance matrix of the random variable $X+Y$. To see why the two matrices are different, use the bilinearity of covariance to see that $$[{\bf K}_{X + Y}]_{ij} = [{\bf K}_{X}]_{ij} + [{\bf K}_{Y}]_{ij} + {\rm cov}(X_i, Y_j) + {\rm cov}(X_j, Y_i)$$ i.e. the cross-covariances are missing from ${\bf K}_X + {\bf K}_Y$ (note I assume $X,Y$ are of equal dimension to ensure that question makes sense). –  Macro Jan 2 '13 at 16:53
Thanks for the response. Am I right to assume then, that if X and Y are independent random variables, then: ${\bf K}_{X+Y}$ = ${\bf K}_X$ + ${\bf K}_Y$ since the covariance of the independent variables is zero? –  nomad2986 Jan 2 '13 at 17:13
Yes, that's correct. –  Macro Jan 2 '13 at 19:35
@whuber: Ok, will do (consider it, that is). I usually prefer to post answers to more involved questions and, when the question is a simple one like this, I typically only post an answer if I can go "above and beyond" somehow, which I don't think I've done, which is why I left it as a comment. In any case, I also encourage you or anyone else (e.g. the OP, since the issue seems to have been clarified) to post their own answer, drawing freely from my comment if desired. Cheers. –  Macro Jan 3 '13 at 15:25

They are different because ${\bf K}_{X} + {\bf K}_Y$ is the sum of two covariance matrices while ${\bf K}_{X+Y}$ is the covariance matrix of the random variable $X+Y$. To see why the two matrices are different, use the bilinearity of covariance to see that
$$[{\bf K}_{X+Y}]_{ij}=[{\bf K}_{X}]_{ij} +[{\bf K}_{Y}]_{ij}+ {\rm cov}(X_i,Y_j)+{\rm cov}(X_j,Y_i)$$
i.e. the cross-covariances are missing from ${\bf K}_{X} + {\bf K}_Y$ (note I assume $X,Y$ are of equal dimension to ensure that question makes sense). So, ${\bf K}_{X+Y}$ is the covariance matrix of $X+Y$ and ${\bf K}_{X} + {\bf K}_Y$ represents the special case where ${\rm cov}(X_i,Y_j)=-{\rm cov}(X_j,Y_i)$ for each pair $(i,j)$, the most notable example being when every element of $X$ is uncorrelated with every element of $Y$.
(+1) I'm glad you turned this into an answer. Note that, strictly speaking, the uncorrelated case is a subcase of $\mathbf K_{X+Y} = \mathbf K_X + \mathbf K_Y$. (Counterexamples, e.g., in the bivariate situation, are easy to construct.) –  cardinal Jan 14 '13 at 18:12