11
$\begingroup$

What are the "best" metrics for covariance matrices, and why? It is clear to me that Frobenius&c are not appropriate, and angle parametrizations have their issues too. Intuitively one might want a compromise between these two, but I would also like to know if there are other aspects to keep in mind and maybe well-established standards.

Common metrics have various drawbacks since they're not natural for covariance matrices, e.g. they often don't especially penalize non PSD matrices or don't behave well w.r.t rank (consider two rotated low-rank covariance ellipsoids: I'd like the same-rank intermediate rotation to have lower distances than the componentwise average, which is not the case with $L_1$ and maybe Frobenius, please correct me here). Also convexity is not always guaranteed. It'd be good seeing these and other issues addressed by a "good" metric.

Here is a good discussion of some issues, one example from network optimization and one from computer vision. And here's a similar question getting some other metrics but without discussion.

$\endgroup$
  • 2
    $\begingroup$ What is the purpose of the metric you seek? For what is the Frobenius metric inappropriate? $\endgroup$ – whuber Jul 1 '13 at 17:32
  • 1
    $\begingroup$ @whuber: I'd like to get an overview in general before imposing too many constraints. My field is quant finance where most people stick to Frobenius for simplicity. Common metrics have various drawbacks since they're not natural for covariance matrices, e.g. they dont especially penalize non PSD matrices and dont behave well w.r.t rank (think of two rotated low-rank covariance ellipsoids: I'd like the same-rank intermediate rotation to have lower distances than the componentwise average, which is not the case with $L_1$ and maybe Frobenius if I'm not mistaken). Added a few links. $\endgroup$ – Quartz Jul 2 '13 at 10:15
  • $\begingroup$ How is that last question you reference "more restricted"? After all, all covariance matrices are symmetric. It seems to be a perfect duplicate. $\endgroup$ – whuber Jul 2 '13 at 11:53
  • 1
    $\begingroup$ That's a good criticism of the other question. May I suggest you edit your question (and title) to reflect the content of your last comment? That will clearly distinguish it from the apparent duplicate and help respondents give you more appropriate replies. (And don't worry about edits to your own question: that is expected; the meta thread is primarily about community editing.) $\endgroup$ – whuber Jul 2 '13 at 12:36
  • 1
    $\begingroup$ @kjetilbhalvorsen That's a provocative sentence! Could you expand in an answer? Or provide an article reference? $\endgroup$ – Sycorax Aug 2 '14 at 0:05
2
$\begingroup$

Well, I don't think there is a good metric or 'the best way' to analyze Covariance matrices. The analysis should be always aligned to your goal. Let's say C is my covariance matrix. The diagonal contains the variance for each computed parameter. So if you're interested in parameter significance then trace(C) is a good start since it's your overall performance.

If you plot your parameter and their significance you can see something like this:

x1 =  1.0 ±  0.1 
x2 = 10.0 ±  5.0
x3 =  5.0 ± 15.0 <-- non-significant parameter

If you're interested in their mutual correlation then such a table might yield something interesting:

x1  1.0
x2  0.9  1.0
x3 -0.3 -0.1  1.0
    x1    x2   x3

Each element is the correlation coefficient between the parameter xi and xj. From the example it's visible that parameter x1 and x2 are highly correlated.

$\endgroup$
1
$\begingroup$

Interesting question, I'm grappling with the same issue at the moment! It depends on how you define 'best', i.e., are you looking for some average single value for the spread, or for the correlation between the data, etc. I found in Press, S.J. (1972): Applied Multivariate Analysis, p. 108 that the generalized variance, defined as the determinant of the covariance matrix, is useful as a single measure for spread. But if it's correlation that you are after, I will need to think futher. Let me know.

$\endgroup$
  • 3
    $\begingroup$ Reference please. $\endgroup$ – Nick Cox Jul 1 '13 at 17:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.