Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

I'm wondering if someone could help to explain the difference between two covariance matrices. Suppose that ${\bf K}_X$ and ${\bf K}_Y$ are two covariance matrices of real random vectors.

What is the difference between ${\bf K}_X+{\bf K}_Y$ and ${\bf K}_{X+Y}$?

share|improve this question
7  
They are different because ${\bf K}_X + {\bf K}_Y$ is the sum of two covariance matrices while ${\bf K}_{X+Y}$ is the covariance matrix of the random variable $X+Y$. To see why the two matrices are different, use the bilinearity of covariance to see that $$ [{\bf K}_{X + Y}]_{ij} = [{\bf K}_{X}]_{ij} + [{\bf K}_{Y}]_{ij} + {\rm cov}(X_i, Y_j) + {\rm cov}(X_j, Y_i) $$ i.e. the cross-covariances are missing from ${\bf K}_X + {\bf K}_Y$ (note I assume $X,Y$ are of equal dimension to ensure that question makes sense). –  Macro Jan 2 '13 at 16:53
1  
Thanks for the response. Am I right to assume then, that if X and Y are independent random variables, then: ${\bf K}_{X+Y}$ = ${\bf K}_X$ + ${\bf K}_Y$ since the covariance of the independent variables is zero? –  nomad2986 Jan 2 '13 at 17:13
    
Yes, that's correct. –  Macro Jan 2 '13 at 19:35
    
@Macro Please consider re-posting your comments as a reply. –  whuber Jan 2 '13 at 21:27
    
@whuber: Ok, will do (consider it, that is). I usually prefer to post answers to more involved questions and, when the question is a simple one like this, I typically only post an answer if I can go "above and beyond" somehow, which I don't think I've done, which is why I left it as a comment. In any case, I also encourage you or anyone else (e.g. the OP, since the issue seems to have been clarified) to post their own answer, drawing freely from my comment if desired. Cheers. –  Macro Jan 3 '13 at 15:25
show 1 more comment

1 Answer 1

While searching through unanswered questions I noticed this one again and decided, in agreement with whuber, that keeping essentially answered questions off of the unanswered tab is higher priority than my own personal preferences about what is "worthy" of answer vs. comment status, so I pasted my comment as an answer.

They are different because ${\bf K}_{X} + {\bf K}_Y$ is the sum of two covariance matrices while ${\bf K}_{X+Y}$ is the covariance matrix of the random variable $X+Y$. To see why the two matrices are different, use the bilinearity of covariance to see that

$$ [{\bf K}_{X+Y}]_{ij}=[{\bf K}_{X}]_{ij} +[{\bf K}_{Y}]_{ij}+ {\rm cov}(X_i,Y_j)+{\rm cov}(X_j,Y_i)$$

i.e. the cross-covariances are missing from ${\bf K}_{X} + {\bf K}_Y$ (note I assume $X,Y$ are of equal dimension to ensure that question makes sense). So, ${\bf K}_{X+Y}$ is the covariance matrix of $X+Y$ and ${\bf K}_{X} + {\bf K}_Y$ represents the special case where ${\rm cov}(X_i,Y_j)=-{\rm cov}(X_j,Y_i)$ for each pair $(i,j)$, the most notable example being when every element of $X$ is uncorrelated with every element of $Y$.

share|improve this answer
1  
(+1) I'm glad you turned this into an answer. Note that, strictly speaking, the uncorrelated case is a subcase of $\mathbf K_{X+Y} = \mathbf K_X + \mathbf K_Y$. (Counterexamples, e.g., in the bivariate situation, are easy to construct.) –  cardinal Jan 14 '13 at 18:12
    
Thanks @cardinal. Corrected. –  Macro Jan 14 '13 at 18:54
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.