5
$\begingroup$

I'm doing some experiments to assess the extend to which MV skewed distributions can affect eigen-vectors (and more specifically Deming regressions).

Suppose $X=(x_i,...,x_n')$ with $x_i \in \mathbb{R}^p$, $1\leq i \leq n$ and for simplicity $p=2$.

I'll be using a (univariate) measure of directional scatter that does not assume symmetricity ($\alpha\in \mathbb{R}^p:||\alpha||=1$):

$$\text{IQR}(X'\alpha)\;\;\;[1]$$

Now, suppose we are trying to find $\alpha^*$:

$$\alpha^*=\underset{\alpha\in \mathbb{R}^p:||\alpha||=1}{\text{Arg.min.}}\;\;\;\text{IQR}(X'\alpha)\;\;\;[2]$$

Then, we have (at least, more suggestions are welcomed) two alternative strategies for finding $\alpha^*$:

  • (a) Sample (pseudo-randomly) a large number $M$ ($M=1000$) of $\alpha_i$'s from the appropriate space$^0$, for each $\alpha_i$ compute $[1]$ and retain as $\alpha^*$ the one that minimizes $[2]$.
  • (b) Use as $\alpha^*$ the eigen-vector corresponding to the last eigen-value of the variance-covariance matrix of $X$.

These two strategies should --for large enough $M$ and $n$-- give comparable results when the $X$'s have an elliptical density, but notice that only strategy (b) requires the elliptical assumption.

Now, i'm trying to measure the difference between the two approaches on skewed $X$ to assess the extend to which skweness affects the estimation of the last eigen-vector --i'm really interested in the consequence of skewness, so i'll only use distributions with finite second moments.

I've tried two (simple) bivariate skewed distributions for the $X$ (the skew normal and the log-normal$^1$) and i don't find large differences $^2$ between approach (a) and (b).

This, and some thinking, lead me to conclude that it's hard to imagine a skewness mechanism that would make (a) very different from (b). I was wondering whether anyone here has a counter-example to that?

$^0$that is, the $\alpha_m$ are the directions of the hyperplanes through $p$ points drawn equiprobably from $1:n$;

$^1$I coordinate-wise $\exp$-ed a MV Gaussian centered at the origin with component-wise variances less than 2 to avoid numerical troubles.

$^2$the median differences over 100 trials are about 10%, which is not very much considering the low efficiency of the IQR.

$\endgroup$
  • $\begingroup$ your welcome: btw, i've added all your suggestions to the text. mmh: thinking of $\alpha^*$ as the direction that minimizes the IQR, and since asymptotically, IQR(x)$\rightarrow$Sd(x) for symmetric $x$'s, then for an elliptical $X$'s there's no ambiguity that $\alpha^*$ is the eigenvector associated with the smallest eigenvalue....not sure this is all that clear $\endgroup$ – user603 Jul 23 '12 at 15:17
  • $\begingroup$ Procrastinator, can you post it as an answer? How do you simulate from a M-Variate SAS-distribution? --i've appended the question for your last remark: i assume that the distributions involved have finite second moment-- $\endgroup$ – user603 Jul 23 '12 at 16:25
  • 1
    $\begingroup$ There are only two steps for fixed $(\mu,\Sigma,\epsilon,\delta)$: (1)Simulate from a MVNormal$(\mu,\Sigma)$, (2) transform each entry using $\sinh[\delta^{-1}\mbox{arcsinh}(x_j)+\epsilon$ *et voilà * and now you have an answer. $\endgroup$ – user10525 Jul 23 '12 at 16:31
5
$\begingroup$

The IQR captures the middle 50% of a distribution. The variance-covariance matrix records second moments, which we can think of as weighted averages of values--with the weights given by the values themselves. Thus a few extreme values will influence the latter while having little or no influence on the former. These two characterizations suggest we can construct a counterexample by combining a distribution that is narrow along the middle 50% of its values with one that has a small proportion of extraordinarily large values. Do this along one axis and along all axes orthogonal to it, do something "tame."

Thus:

set.seed(17)
n <- 100               # Number of small values in the x direction
m <- 80                # Number of potentially large values in the x direction
x <- c(runif(n, min=-1/2, max=1/2), 1/rnorm(m))
y <- rnorm(n+m, sd=2)  # Intermediate values go in the y direction
data <- cbind(x,y)     # Create (x,y) data
plot(data)
cov(data)
eigen(cov(data))
c(IQR(x), IQR(y))

Plot

The eigenvectors, which are close to $(-1,0)$ and $(0,-1)$, show that the principal directions are essentially $(1,0)$ and $(0,1)$. The eigenvalues indicate there is much less variance along the latter direction (4.1 vs 38.2). However, the IQR along the first principal direction (0.86) is only about a third of that along the second direction (2.6), exactly as planned. Although this does not exactly determine $\alpha^*$, I hope it makes it clear why $\alpha^*$ and the smallest principal direction can be arbitrarily far apart.

These data can be made "skew" in the technical sense (of a substantial standardized third central moment) simply by extending some of the extreme $x$ values on one side of the plot compared to the other. This will barely change the eigenvectors or the IQRs.

$\endgroup$
  • 1
    $\begingroup$ This answer is correct (and really awesome) but it uses a feature of the particular estimator i used in the question (the IQR) --it's robustness-- that won't be binding in the real application. So i will repost a modified version of this question using an alternative estimator of scatter (the columnwise range). $\endgroup$ – user603 Jul 23 '12 at 17:42
  • 1
    $\begingroup$ OK, but the robustness was not essential to construction of the counterexample: what mattered was that the two alternative measures of dispersion can be controlled in a quasi-independent way by suitable modifications of the data. With a range, for instance, you can fix the range at any desired value and, by changing all the other data values, vary the variance anywhere between an infimum of $0$ (for arbitrarily large amounts of data) and a maximum of the one-quarter the square of the range. This should permit constructing similar counterexamples. $\endgroup$ – whuber Jul 23 '12 at 18:23
  • 1
    $\begingroup$ Great example. i'm still not sure why your answer works, but, indeed, i have been able to use the template your provided to build a counter example for the range case. If i still don't get the intuition by tomorrow i will perhaps resort to asking a separate question about it. Thanks for coming up with it though. $\endgroup$ – user603 Jul 23 '12 at 23:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.