Distance invariant approaches to find the "main" difference between two distributions in R$^n$ If I have a population of vectors in R$^n$ and some special subset has a different distribution, I could try and use PCA to describe the main axes of these distributions and, if they are aligned differently, that would give me a quick way to describe the difference in orientation between the two. However PCA is dependent on the scale of the components of the vectors. So what is an analagous approach that can cleanly describe the "main" difference in the the two populations given that they have different covariance matrices, but doesn't depend on the scaling given to the componenents? It is possible the answer is more easily stated in terms of the two Mahalanobis distances one gets from the two distributions.
Anyway, I'm sure there is a standard way to describe the "main" difference between two such distributions, but I don't know what it is. 
P.S. Yes, assume they have the same mean.
 A: Here is what I have come up with.
The short version is that if $A$ and $B$ are the covariance matrices of two distributions then the eigenvectors and eigenvalues of $AB^{-1}$ give such a characterization. Basically an eigenvector $u$ with eigenvalue $\lambda$ is a direction in which distribution $A$ is $\sqrt{\lambda}$ times as thick as distribution $B$. So if $\sqrt{\lambda}$ is large then $u$ is a direction where the $A$ distribution is thick and the $B$ distribution is thin and if $\sqrt{\lambda}$ is small then the opposite is true, $B$ is thick in direction $u$ and $A$ is thin in that direction.
Any two such eigenvectors are orthogonal in either mahalanobis distance (the one for $A$ or the one for $B$) and if $u$ is one of these eigenvectors with eigenvalue $\lambda$ then the mahalanobis length of $u$ using the $B$ covariance matrix is $\sqrt{\lambda}$ times the mahalanobis length of $u$ using the $A$ covariance matrix.
The largest eigenvalue corresponds to the direction in which A has the most variance relative to B (i.e. where A is the thickest compared to B) and the smallest corresponds to the direction in which B has the most variance relative to A.
The Math
If $AB^{-1} u = \lambda u$ then $B^{-1}u = \lambda A^{-1}u$. The $A$ distribution mahalanobis inner product is $<x,y>=x^tA^{-1}y$ and the $B$ distribution mahalanobis inner product is $<x,y>=x^tB^{-1}y$. From these it is easy to prove that any two eigenvectors with different eigenvalues are orthogonal using either inner product and that $\|u\|_B = \sqrt{\lambda} \|u\|_A$ where $\|u\|_A$ is the $A$ distribution mahalanobis length of $u$ and $\|u\|_B$ is the $B$ distribution mahalanobis length of $u$ (note $\lambda$ is always positive). 
The eigenvectors form an orthogonal basis using either mahalanobis inner product.
Explanation
The point is given any two zero mean multivariable normal distributions in R^n, there is a linear change of coordinates so that BOTH of those normals are independent on each component seperately, i.e. after this change of coordinates both of our normal distributions on R^n are really just n independent one dimensional normals. The eigenvalues described above are exactly the n ratios of the variances one gets from those n pairs of distributions on R.
