Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

are there any measures of similarity or distance between two symmetric covariance matrices (both having the same dimension)?

I am thinking here of analogues to KL divergence of two probability distributions or the Euclidean distance between vectors except applied to matrices. I imagine there would be quite a few similarity measurements.

Ideally I would also like to test the null-hypothesis that two covariance matrices are identical.

share|improve this question
2  
the answers to this question: quant.stackexchange.com/q/121/108 may be of some use. –  shabbychef Aug 23 '11 at 3:41
    
excellent question and answer on the link - thanks - yes this is where I was going :) –  Quant Guy Aug 23 '11 at 5:21
add comment

4 Answers

up vote 7 down vote accepted

You can use any of the norms $\| A-B \|_p $ (see Wikipedia on a variety of norms; note that the sum of squared distances, $\sum_{i,j} (a_{ij}-b_{ij})^2$, is called Frobenius norm, and is different from $L_2$ norm, which is the square root of the largest eigenvalue of $(A-B)^2$, although of course they would generate the same topology). The K-L distance between the two normal distributions with the same means (say zero) and the two specific covariance matrices is also available in Wikipedia as $\frac12 [ \mbox{tr} (A^{-1}B) - \mbox{ln}( |B|/|A| ) ]$.

Edit: if one of the matrices is a model-implied matrix, and the other is the sample covariance matrix, then of course you can form a likelihood ratio test between the two. My personal favorite collection of such tests for simple structures is given in Rencher (2002) Methods of Multivariate Analysis. More advanced cases are covered in covariance structure modeling, on which a reasonable starting point is Bollen (1989) Structural Equations with Latent Variables.

share|improve this answer
    
i have a problem with $1/2(\verb+tr+(A^{-1}B)-\log(|B|/|A|))$: it doesn't give the same value if you permute $A$ and $B$ ( a real distance should be symmetric). –  user603 Aug 23 '11 at 6:49
    
i have a problem with $(A-B)^2$: it is not affine equivariant (if you rotate the matrices, there distance changes!). Furthermore, you should somehow scale your matrices (they might be measured in very different units), also, it is only natural to require that the distance between two covariance matrices be the same as the distance between the corresponding correlation matrices: so I suggest $(A\det(A)^{-1/p}-B\det(B)^{-1/p})^2$. –  user603 Aug 23 '11 at 6:54
    
First, K-L is not a real distance, and that's a well known fact. Second, if the matrices are measured in different units, they cannot be equal. –  StasK Aug 23 '11 at 13:49
add comment

there are two further that i know of. Denote $\varSigma_1$ and $\varSigma_2$ your matrices both of dimension $p$.

  1. Cond number: $\log(\lambda_1)-\log(\lambda_p)$ where $\lambda_1$ ($\lambda_p$) is the largest (smallest) eigenvalue of $\varSigma^*$, where $\varSigma^*$ is defined as: $\varSigma^*:=\varSigma_1^{-1/2}\varSigma_2\varSigma_1^{-1/2}$

  2. Det ratio: $\log(\det(\varSigma^{**})/\sqrt{\det(\varSigma_2)*\det(\varSigma_1)})$ where $\varSigma^{**}=(\varSigma_1+\varSigma_2)/2$.

The first one focuses on the maximum difference (think of it as a matrix version of the sup norm) and the second one on average differences (think of it as the matrix equivalent of the 2 norm).

share|improve this answer
add comment

The covariance matrix distance is used for tracking objects in Computer Vision.

The currently used metric is described in the article: "A metric for covariance matrices", by Förstner and Moonen.

share|improve this answer

We're looking for long answers that provide some explanation and context. Don't just give a one-line answer; explain why your answer is right, ideally with citations. Answers that don't include explanations may be removed.

add comment

A measure introduced by Herdin et al. (2005) is $$d = 1 - \frac{\text{trace(R1} \cdot \text{R2)}}{\text{norm(R1)} \cdot \text{norm(R2)}}$$ where the norm is the Frobenious norm. The paper is available here.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.