What is the difference between the sum of two covariance matrices and the covariance matrix of the sum of two variables?

I'm wondering if someone could help to explain the difference between two covariance matrices. Suppose that ${\bf K}_X$ and ${\bf K}_Y$ are two covariance matrices of real random vectors.

What is the difference between ${\bf K}_X+{\bf K}_Y$ and ${\bf K}_{X+Y}$?

• They are different because ${\bf K}_X + {\bf K}_Y$ is the sum of two covariance matrices while ${\bf K}_{X+Y}$ is the covariance matrix of the random variable $X+Y$. To see why the two matrices are different, use the bilinearity of covariance to see that $$[{\bf K}_{X + Y}]_{ij} = [{\bf K}_{X}]_{ij} + [{\bf K}_{Y}]_{ij} + {\rm cov}(X_i, Y_j) + {\rm cov}(X_j, Y_i)$$ i.e. the cross-covariances are missing from ${\bf K}_X + {\bf K}_Y$ (note I assume $X,Y$ are of equal dimension to ensure that question makes sense). Jan 2 '13 at 16:53
• Thanks for the response. Am I right to assume then, that if X and Y are independent random variables, then: ${\bf K}_{X+Y}$ = ${\bf K}_X$ + ${\bf K}_Y$ since the covariance of the independent variables is zero? Jan 2 '13 at 17:13
• Yes, that's correct. Jan 2 '13 at 19:35
– whuber
Jan 2 '13 at 21:27
• @Macro Please bear in mind that comments are intended as vehicles to request clarification (and to provide clarification), not as a means to post answers to "simple" questions. No matter how simple the question or the answer, please use the answering mechanism to post your answers. That will help reduce our rate of "unanswered" questions, something that is of concern to the SE team.
– whuber
Jan 3 '13 at 15:30

They are different because ${\bf K}_{X} + {\bf K}_Y$ is the sum of two covariance matrices while ${\bf K}_{X+Y}$ is the covariance matrix of the random variable $X+Y$. To see why the two matrices are different, use the bilinearity of covariance to see that
$$[{\bf K}_{X+Y}]_{ij}=[{\bf K}_{X}]_{ij} +[{\bf K}_{Y}]_{ij}+ {\rm cov}(X_i,Y_j)+{\rm cov}(X_j,Y_i)$$
i.e. the cross-covariances are missing from ${\bf K}_{X} + {\bf K}_Y$ (note I assume $X,Y$ are of equal dimension to ensure that question makes sense). So, ${\bf K}_{X+Y}$ is the covariance matrix of $X+Y$ and ${\bf K}_{X} + {\bf K}_Y$ represents the special case where ${\rm cov}(X_i,Y_j)=-{\rm cov}(X_j,Y_i)$ for each pair $(i,j)$, the most notable example being when every element of $X$ is uncorrelated with every element of $Y$.
• (+1) I'm glad you turned this into an answer. Note that, strictly speaking, the uncorrelated case is a subcase of $\mathbf K_{X+Y} = \mathbf K_X + \mathbf K_Y$. (Counterexamples, e.g., in the bivariate situation, are easy to construct.) Jan 14 '13 at 18:12