9
$\begingroup$

What is a good measure of spread for a multivariate normal distribution?

I was thinking about using an average of the component standard deviations; perhaps the trace of the covariance matrix divided by the number of dimensions, or a version of that. Is that any good?

Thanks

Improve this question
$\endgroup$
3
  • 1
    $\begingroup$ as such, the spread of multivariate gaussian doesn't make sense. However, depending on your needs, there might exists approaches to answer your question. Trace of the matrix is one of the many ways, but you would be ignoring correlations, which may make a huge difference. Eigen values, PCA, etc. might be much better. Therefore, could you please elaborate on your needs? $\endgroup$
    – suncoolsu
    Jul 7 '11 at 10:41
  • $\begingroup$ As such, I want an analog of the standard deviation to a multi-dimensional space. Yes, the trace would ignore the correlations, which is what I fear. Having said that, this does not need to be mathematically exact. Basically, a good indication of spread would be the hypervolume size of the hyperellipse defined by 1 std. deviation from the mean. But a nice, handy formula without deriving the exact volume would be much appreciated. $\endgroup$
    – Kristian D'Amato
    Jul 7 '11 at 10:48
  • $\begingroup$ Seems like PCA could answer your question. $\endgroup$
    – suncoolsu
    Jul 7 '11 at 10:51
14
$\begingroup$

What about the determinant of the sample variance-covariance matrix: a measure of the squared volume enclosed by the matrix within the space of dimension of the measurement vector. Also, an often used scale invariant version of that measure is the determinant of the sample correlation matrix: the volume of the space occupied within the dimensions of the measurement vector.

Improve this answer
$\endgroup$
6
  • 1
    $\begingroup$ +1 Yes, the determinants are directly related to the "hypervolume...of the ellipse defined by 1 sd from the mean." $\endgroup$
    – whuber ♦
    Jul 7 '11 at 13:53
  • $\begingroup$ So that's the determinant of the covariance matrix, right? $\endgroup$
    – Kristian D'Amato
    Jul 7 '11 at 14:29
  • 2
    $\begingroup$ @Kristian The square root of the determinant of the covariance matrix tells you the hypervolume, incorporating both shape (correlation) and size (standard deviation) information. It is the product of the standard deviations of the principal components. The determinant of the correlation matrix is basically a shape factor only, ranging from 0 for degenerate distributions up to 1 when all components are uncorrelated. $\endgroup$
    – whuber ♦
    Jul 7 '11 at 18:42
  • $\begingroup$ @whuber, what if I'd like to have a separate measurement of shape and size? (I'm actually interested in the size only, I think.) $\endgroup$
    – Atcold
    Feb 12 '17 at 18:24
  • $\begingroup$ @Atcold You would need to establish a quantitative definition of "size". This would be equivalent to establishing what a unit-size distribution is for each given shape. (By definition, "shape" is whatever properties a distribution may have that are unchanged by translation or rescaling.) There are innumerable ways to do that, so ultimately the issue comes down to choosing a suitable definition for your particular analysis. This is one reason there cannot be a universal definition of size (or "spread") for any distribution family that comprises multiple shapes. $\endgroup$
    – whuber ♦
    Feb 12 '17 at 18:28
3
$\begingroup$

I would go with either trace or determinant with a preference towards trace depending on the application. They're both good in that they're invariant to representation and have clear geometric meanings.

I think there is a good argument to be made for Trace over Determinant.

The determinant effectively measures the volume of the uncertainty ellipsoid. If there is any redundancy in your system however then the covariance will be near-singular (the ellipsoid is very thin in one direction) and then the determinant/volume will be near-zero even if there is a lot of uncertainty/spread in the other directions. In a moderate to high-dimensional setting this occurs very frequently

The trace is geometrically the sum of the lengths of the axes and is more robust to this sort of situation. It will have a non-zero value even if some of the directions are certain.

Additionally, the trace is generally much easier to compute.

Improve this answer
$\endgroup$
3
  • $\begingroup$ +1 Good points. This gets me thinking: any symmetric function of the $n$ eigenvalues would qualify as "good." All such polynomial functions are polynomials in the $n$ elementary symmetric functions, which include the determinant and the trace. $\endgroup$
    – whuber ♦
    Jul 15 '11 at 20:53
  • $\begingroup$ Yes, the sum (trace) isn't necessarily the best way to go. You're right that you could imagine lots of mixtures here depending on the application. I wonder if there is some standard family of functions that would be good here.... $\endgroup$
    – MRocklin
    Jul 15 '11 at 20:56
  • $\begingroup$ @MR I'm not aware of anybody attempting to use a single statistic to compute the spread of a multivariate normal distribution (except, of course, when independence of all components is assumed). This leads me to believe there may be no such standard family. $\endgroup$
    – whuber ♦
    Jul 18 '11 at 12:44
1
$\begingroup$

Another (closely related) quantity is the entropy of the distribution: for a multivariate Gaussian this is the log of the determinant of the covariance matrix, or

$\frac{1}{2} \log |(2\pi e)\Lambda|$

where $\Lambda$ is the covariance matrix. The advantage of this choice is that it can be compared to the "spread" of points under other (e.g., non-Gaussian) distributions.

(If we want to get technical, this is the differential entropy of a Gaussian).

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.