6
$\begingroup$

What's a good measure of spread in a multidimensional space?

In a single dimension variance would be the measure I need, but in a multidimensional space I need more than just variances. Note that in a single dimension, I need something different than the range covered by the data points.

Consider a hypercube representing the space of all possible data points (all attributes are limited to values in the interval $(0,1)$). I need a measure of spread which is optimal when all corners of the cube are equally populated (and there are no other data points within the cube). When only two opposed corners would be populated, all variances would be maximal, but this is not what I need. Also a population which covers all corners, but also contains points more toward the middle of the cube should have a lower spread.

My first hunch would be to sum up all values on the main diagonal of the covariance matrix, and substract all other values (all proper covariances). This idea is very ad hoc, however, and I don't know whether this is thinking in the right direction.

Please help me find a good measure of spread/variance in a multidimensional space.

$\endgroup$
  • 1
    $\begingroup$ If your observations are symmetrically distributed about a center, then, the determinant of the covariance matrix of your observations is the measure you want. $\endgroup$ – user603 Oct 8 '13 at 13:08
  • 1
    $\begingroup$ See answer to this question. $\endgroup$ – user603 Jul 25 '16 at 21:57
4
$\begingroup$

It depends what you want to measure, exactly, but here are two suggestions.

Let $\Sigma$ denote the covariance matrix, and note that the trace of $\Sigma$ is equal to the sum of the eigenvalues, while the determinant is equal to their product. This means that $$\text{Tr}(\Sigma)=\sum \lambda_i$$ will give you the "total variance", while $$\sqrt{|\Sigma|}=\prod\sqrt{\lambda_i}$$ will give you the volume of the hyperrectangle whose side lengths are determined by square roots of the eigenvalues (i.e. the standard deviation in each orthogonal direction). So, if you are looking for a measure of the volume covered by the distribution, then you should go with $\sqrt{|\Sigma|}$.

Note that this also gives some intuition behind the fact that $\sqrt{|\Sigma|}$ appears in the denominator of the density function for the multivariate normal distribution: $$\frac{1}{\sqrt{(2\pi)^k|\Sigma|}}e^{-\frac{1}{2}(x-\mu)^t\Sigma^{-1}(x-\mu)}$$

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Can you maybe supply a source or reference for the equation $\sqrt{|\Sigma|}=\prod\sqrt{\lambda_i}$? $\endgroup$ – Angelorf May 15 '14 at 11:04
  • $\begingroup$ @Angelorf It's straightforward. The determinant of a matrix is the the product of its eigenvalues (math.stackexchange.com/q/507641/149510). $\endgroup$ – WYSIWYG Jun 24 '19 at 15:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.