# Eigen-vectors and skewness

I'm doing some experiments to assess the extend to which MV skewed distributions can affect eigen-vectors (and more specifically Deming regressions).

Suppose $X=(x_i,...,x_n')$ with $x_i \in \mathbb{R}^p$, $1\leq i \leq n$ and for simplicity $p=2$.

I'll be using a (univariate) measure of directional scatter that does not assume symmetricity ($\alpha\in \mathbb{R}^p:||\alpha||=1$):

$$\text{IQR}(X'\alpha)\;\;\;[1]$$

Now, suppose we are trying to find $\alpha^*$:

$$\alpha^*=\underset{\alpha\in \mathbb{R}^p:||\alpha||=1}{\text{Arg.min.}}\;\;\;\text{IQR}(X'\alpha)\;\;\;[2]$$

Then, we have (at least, more suggestions are welcomed) two alternative strategies for finding $\alpha^*$:

• (a) Sample (pseudo-randomly) a large number $M$ ($M=1000$) of $\alpha_i$'s from the appropriate space$^0$, for each $\alpha_i$ compute $[1]$ and retain as $\alpha^*$ the one that minimizes $[2]$.
• (b) Use as $\alpha^*$ the eigen-vector corresponding to the last eigen-value of the variance-covariance matrix of $X$.

These two strategies should --for large enough $M$ and $n$-- give comparable results when the $X$'s have an elliptical density, but notice that only strategy (b) requires the elliptical assumption.

Now, i'm trying to measure the difference between the two approaches on skewed $X$ to assess the extend to which skweness affects the estimation of the last eigen-vector --i'm really interested in the consequence of skewness, so i'll only use distributions with finite second moments.

I've tried two (simple) bivariate skewed distributions for the $X$ (the skew normal and the log-normal$^1$) and i don't find large differences $^2$ between approach (a) and (b).

This, and some thinking, lead me to conclude that it's hard to imagine a skewness mechanism that would make (a) very different from (b). I was wondering whether anyone here has a counter-example to that?

$^0$that is, the $\alpha_m$ are the directions of the hyperplanes through $p$ points drawn equiprobably from $1:n$;

$^1$I coordinate-wise $\exp$-ed a MV Gaussian centered at the origin with component-wise variances less than 2 to avoid numerical troubles.

$^2$the median differences over 100 trials are about 10%, which is not very much considering the low efficiency of the IQR.

• your welcome: btw, i've added all your suggestions to the text. mmh: thinking of $\alpha^*$ as the direction that minimizes the IQR, and since asymptotically, IQR(x)$\rightarrow$Sd(x) for symmetric $x$'s, then for an elliptical $X$'s there's no ambiguity that $\alpha^*$ is the eigenvector associated with the smallest eigenvalue....not sure this is all that clear – user603 Jul 23 '12 at 15:17
• Procrastinator, can you post it as an answer? How do you simulate from a M-Variate SAS-distribution? --i've appended the question for your last remark: i assume that the distributions involved have finite second moment-- – user603 Jul 23 '12 at 16:25
• There are only two steps for fixed $(\mu,\Sigma,\epsilon,\delta)$: (1)Simulate from a MVNormal$(\mu,\Sigma)$, (2) transform each entry using $\sinh[\delta^{-1}\mbox{arcsinh}(x_j)+\epsilon$ *et voilà * and now you have an answer. – user10525 Jul 23 '12 at 16:31

The IQR captures the middle 50% of a distribution. The variance-covariance matrix records second moments, which we can think of as weighted averages of values--with the weights given by the values themselves. Thus a few extreme values will influence the latter while having little or no influence on the former. These two characterizations suggest we can construct a counterexample by combining a distribution that is narrow along the middle 50% of its values with one that has a small proportion of extraordinarily large values. Do this along one axis and along all axes orthogonal to it, do something "tame."

Thus:

set.seed(17)
n <- 100               # Number of small values in the x direction
m <- 80                # Number of potentially large values in the x direction
x <- c(runif(n, min=-1/2, max=1/2), 1/rnorm(m))
y <- rnorm(n+m, sd=2)  # Intermediate values go in the y direction
data <- cbind(x,y)     # Create (x,y) data
plot(data)
cov(data)
eigen(cov(data))
c(IQR(x), IQR(y))


The eigenvectors, which are close to $(-1,0)$ and $(0,-1)$, show that the principal directions are essentially $(1,0)$ and $(0,1)$. The eigenvalues indicate there is much less variance along the latter direction (4.1 vs 38.2). However, the IQR along the first principal direction (0.86) is only about a third of that along the second direction (2.6), exactly as planned. Although this does not exactly determine $\alpha^*$, I hope it makes it clear why $\alpha^*$ and the smallest principal direction can be arbitrarily far apart.

These data can be made "skew" in the technical sense (of a substantial standardized third central moment) simply by extending some of the extreme $x$ values on one side of the plot compared to the other. This will barely change the eigenvectors or the IQRs.

• This answer is correct (and really awesome) but it uses a feature of the particular estimator i used in the question (the IQR) --it's robustness-- that won't be binding in the real application. So i will repost a modified version of this question using an alternative estimator of scatter (the columnwise range). – user603 Jul 23 '12 at 17:42
• OK, but the robustness was not essential to construction of the counterexample: what mattered was that the two alternative measures of dispersion can be controlled in a quasi-independent way by suitable modifications of the data. With a range, for instance, you can fix the range at any desired value and, by changing all the other data values, vary the variance anywhere between an infimum of $0$ (for arbitrarily large amounts of data) and a maximum of the one-quarter the square of the range. This should permit constructing similar counterexamples. – whuber Jul 23 '12 at 18:23
• Great example. i'm still not sure why your answer works, but, indeed, i have been able to use the template your provided to build a counter example for the range case. If i still don't get the intuition by tomorrow i will perhaps resort to asking a separate question about it. Thanks for coming up with it though. – user603 Jul 23 '12 at 23:14