0
$\begingroup$

This question is a follow-up to: Measures of multidimensional spread or variance

I am interested in getting a multivariate measure of total variance in numeric data. Imagine we have a covariance matrix. We could get the sum of the diagonal, which would be summing up the variance across each column. But this doesn't account for covariation between variables: Holding variances equal, a higher covariance between variables means the points take up less geometric space.

In the link above, two measures are proposed. One in a comment, and one in the accepted answer:

  • Get the determinant of the covariance matrix. This accounts for variance in each variable, then discounts it for how much they covary.

  • But the one that makes more sense to me is the answer. The answerer says that: the square root of the eigenvalues are the sides of the p-dimensional hyper-rectangle. Which means that multiplying these all together will get you the volume of said rectangle, which is exactly what I want.

Can someone present a citation or illustrate why is it that the square root of the eigenvalues are the sides of the rectangle? I do better with simulated data than I do with proofs, so I included some code. I want to generalize this to a p-dimensional surface, but for simplicity let's look at it with just two variables.

I simulate data where predictors are correlated at .8 or .0. I fix all variances of each variable to be the same.

library(mvtnorm)
set.seed(1839)
d8 <- rmvnorm(50000, sigma = matrix(c(1, .8, .8, 1), 2))
d0 <- rmvnorm(50000, sigma = matrix(c(1, .0, .0, 1), 2))
cov8 <- cov(d8)
cov0 <- cov(d0)

So the sum of the diagonals of each covariance matrices are basically the same:

> sum(diag(cov8))
[1] 2.013285
> sum(diag(cov0))
[1] 2.005801

But we can see that the uncorrelated data take up much more space in the same two-dimensional surface by calling plot on each of the datasets:

enter image description here

enter image description here

Both of the methods above show me that more spread is in the second graph than the first:

> det(cov8); prod(sqrt(eigen(cov8)$values))
[1] 0.3626525
[1] 0.6022064
> det(cov0); prod(sqrt(eigen(cov0)$values))
[1] 1.005795
[1] 1.002893

But what is the logic here, geometrically? Here are the square roots of the eigenvalues:

> sqrt(eigen(cov8)$values)
[1] 1.3465838 0.4472105
> sqrt(eigen(cov0)$values)
[1] 1.0033179 0.9995769

I don't see how these "lengths of the sides of the rectangle" map onto the plots above?

The answer here also appears to be informative. But I'm a mere quantitative social scientist and need more hand-holding when it comes to the math.

$\endgroup$
4
  • 1
    $\begingroup$ The "rectangle" is a red herring because it isn't relevant. That term should be replaced by "ellipsoid," with "sides" replaced by "semi-axes" and "(hyper)volume of" replaced by "proportional to the hypervolume of." For insight into this, see stats.stackexchange.com/questions/62092 or (for two dimensions only) stats.stackexchange.com/questions/71260. $\endgroup$ – whuber Apr 15 '20 at 22:39
  • $\begingroup$ @whuber So would you propose using a different metric here? Or is it just that the interpretation/explanation bit is off? $\endgroup$ – Mark White Apr 15 '20 at 22:42
  • $\begingroup$ I'm not proposing any particular metric. (Using a single number to characterize the association of many variables always strikes me as overly limited.) I'm just giving some intuition for the (hyper)volume that is naturally associated with this situation. $\endgroup$ – whuber Apr 15 '20 at 22:43
  • 1
    $\begingroup$ @whuber Got it—I see you updated your comment. I'll look at those links. I need one particular number here, even if its limited, but I want to make sure I understand the ways in which it is limited, so I'm diving into the logic behind it. $\endgroup$ – Mark White Apr 15 '20 at 22:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.