This question is a follow-up to: Measures of multidimensional spread or variance
I am interested in getting a multivariate measure of total variance in numeric data. Imagine we have a covariance matrix. We could get the sum of the diagonal, which would be summing up the variance across each column. But this doesn't account for covariation between variables: Holding variances equal, a higher covariance between variables means the points take up less geometric space.
In the link above, two measures are proposed. One in a comment, and one in the accepted answer:
Get the determinant of the covariance matrix. This accounts for variance in each variable, then discounts it for how much they covary.
But the one that makes more sense to me is the answer. The answerer says that: the square root of the eigenvalues are the sides of the p-dimensional hyper-rectangle. Which means that multiplying these all together will get you the volume of said rectangle, which is exactly what I want.
Can someone present a citation or illustrate why is it that the square root of the eigenvalues are the sides of the rectangle? I do better with simulated data than I do with proofs, so I included some code. I want to generalize this to a p-dimensional surface, but for simplicity let's look at it with just two variables.
I simulate data where predictors are correlated at .8 or .0. I fix all variances of each variable to be the same.
library(mvtnorm)
set.seed(1839)
d8 <- rmvnorm(50000, sigma = matrix(c(1, .8, .8, 1), 2))
d0 <- rmvnorm(50000, sigma = matrix(c(1, .0, .0, 1), 2))
cov8 <- cov(d8)
cov0 <- cov(d0)
So the sum of the diagonals of each covariance matrices are basically the same:
> sum(diag(cov8))
[1] 2.013285
> sum(diag(cov0))
[1] 2.005801
But we can see that the uncorrelated data take up much more space in the same two-dimensional surface by calling plot on each of the datasets:
Both of the methods above show me that more spread is in the second graph than the first:
> det(cov8); prod(sqrt(eigen(cov8)$values))
[1] 0.3626525
[1] 0.6022064
> det(cov0); prod(sqrt(eigen(cov0)$values))
[1] 1.005795
[1] 1.002893
But what is the logic here, geometrically? Here are the square roots of the eigenvalues:
> sqrt(eigen(cov8)$values)
[1] 1.3465838 0.4472105
> sqrt(eigen(cov0)$values)
[1] 1.0033179 0.9995769
I don't see how these "lengths of the sides of the rectangle" map onto the plots above?
The answer here also appears to be informative. But I'm a mere quantitative social scientist and need more hand-holding when it comes to the math.
