Measures of similarity or distance between two covariance matrices - Cross Validated most recent 30 from stats.stackexchange.com 2019-10-23T00:46:18Z https://stats.stackexchange.com/feeds/question/14673 https://creativecommons.org/licenses/by-sa/4.0/rdf https://stats.stackexchange.com/q/14673 28 Measures of similarity or distance between two covariance matrices Ram Ahluwalia https://stats.stackexchange.com/users/8101 2011-08-23T02:40:04Z 2019-03-19T10:47:55Z <p>Are there any measures of similarity or distance between two symmetric covariance matrices (both having the same dimensions)?</p> <p>I am thinking here of analogues to KL divergence of two probability distributions or the Euclidean distance between vectors except applied to matrices. I imagine there would be quite a few similarity measurements.</p> <p>Ideally I would also like to test the null hypothesis that two covariance matrices are identical.</p> https://stats.stackexchange.com/questions/14673/-/14676#14676 21 Answer by StasK for Measures of similarity or distance between two covariance matrices StasK https://stats.stackexchange.com/users/5739 2011-08-23T03:18:09Z 2016-07-29T13:48:39Z <p>You can use any of the norms $\| A-B \|_p$ (see <a href="http://en.wikipedia.org/wiki/Matrix_norm" rel="noreferrer">Wikipedia</a> on a variety of norms; note that the square-root of the sum of squared distances, $\sqrt{\sum_{i,j} (a_{ij}-b_{ij})^2}$, is called Frobenius norm, and is different from $L_2$ norm, which is the square root of the largest eigenvalue of $(A-B)^2$, although of course they would generate the same topology). The K-L distance between the two normal distributions with the same means (say zero) and the two specific covariance matrices is also available in <a href="http://en.wikipedia.org/wiki/Multivariate_normal_distribution#Kullback.E2.80.93Leibler_divergence" rel="noreferrer">Wikipedia</a> as $\frac12 [ \mbox{tr} (A^{-1}B) - \mbox{ln}( |B|/|A| ) ]$.</p> <p><strong>Edit:</strong> if one of the matrices is a model-implied matrix, and the other is the sample covariance matrix, then of course you can form a likelihood ratio test between the two. My personal favorite collection of such tests for simple structures is given in <a href="http://rads.stackoverflow.com/amzn/click/0471418897" rel="noreferrer">Rencher (2002) <em>Methods of Multivariate Analysis</em></a>. More advanced cases are covered in covariance structure modeling, on which a reasonable starting point is <a href="http://rads.stackoverflow.com/amzn/click/0471011711" rel="noreferrer">Bollen (1989) <em>Structural Equations with Latent Variables</em></a>.</p> https://stats.stackexchange.com/questions/14673/-/14680#14680 7 Answer by user603 for Measures of similarity or distance between two covariance matrices user603 https://stats.stackexchange.com/users/603 2011-08-23T06:40:53Z 2016-07-29T13:11:29Z <p>Denote $\varSigma_1$ and $\varSigma_2$ your matrices both of dimension $p$.</p> <ol> <li>Cond number: $\log(\lambda_1)-\log(\lambda_p)$ where $\lambda_1$ ($\lambda_p$) is the largest (smallest) eigenvalue of $\varSigma^*$, where $\varSigma^*$ is defined as: $\varSigma^*:=\varSigma_1^{-1/2}\varSigma_2\varSigma_1^{-1/2}$</li> </ol> <p>Edit: I edited out the second of the two proposals. I think I had misunderstood the question. The proposal based on condition numbers is used in robust statistics a lot to assess quality of fit. An old source I could find for it is:</p> <blockquote> <p>Yohai, V.J. and Maronna, R.A. (1990). The Maximum Bias of Robust Covariances. Communications in Statistics–Theory and Methods, 19, 3925–2933.</p> </blockquote> <p>I had originally included the Det ratio measure:</p> <blockquote> <ol start="2"> <li>Det ratio: $\log(\det(\varSigma^{**})/\sqrt{\det(\varSigma_2)*\det(\varSigma_1)})$ where $\varSigma^{**}=(\varSigma_1+\varSigma_2)/2$.</li> </ol> </blockquote> <p>which would be the <a href="https://en.wikipedia.org/wiki/Bhattacharyya_distance" rel="nofollow">Bhattacharyya distance</a> between two Gaussian distributions having the same location vector. I must have originally read the question as pertaining to a setting where the two covariances were coming from samples from populations assumed to have equal means. </p> https://stats.stackexchange.com/questions/14673/-/29680#29680 4 Answer by Andres Romero for Measures of similarity or distance between two covariance matrices Andres Romero https://stats.stackexchange.com/users/11714 2012-06-02T10:51:57Z 2019-03-19T10:47:55Z <p>The covariance matrix distance is used for tracking objects in Computer Vision. </p> <p>The currently used metric is described in the article: <a href="http://www.ipb.uni-bonn.de/pdfs/Forstner1999Metric.pdf" rel="nofollow noreferrer">"A metric for covariance matrices"</a>, by Förstner and Moonen.</p> https://stats.stackexchange.com/questions/14673/-/100498#100498 7 Answer by davidc for Measures of similarity or distance between two covariance matrices davidc https://stats.stackexchange.com/users/30885 2014-05-29T14:49:52Z 2019-03-19T10:47:46Z <p>A measure introduced by <a href="https://www.researchgate.net/publication/4194743_Correlation_matrix_distance_a_meaningful_measure_for_evaluation_of_non-stationary_MIMO_channels" rel="nofollow noreferrer">Herdin (2005) Correlation Matrix Distance, a Meaningful Measure for Evaluation of Non-Stationary MIMO Channels</a> is <span class="math-container">$$d = 1 - \frac{\text{tr}(R_1 \cdot R_2)}{\|R_1\| \cdot \|R_2\|},$$</span> where the norm is the Frobenius norm.</p>