Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

This is really a side project of mine ... while writing on a paper on something totally different! I read (part of ) the excellent paper


This shows among other things that misbehavior of PCA is connected to "crossings of eigenvalues". If you are interested in estimating the eigenvector corresponding to the largest (model) eigenvalue, then if the largest sample eigenvalue frequently comes from a smaller (model) eigenvalue, which is a "crossing", the corresponding sample eigenvector do not correspond to what we try to estimate, and PCA breaks down.

So, given a sample, it would be interesting to estimate (for example, using the bootstrap) the probability of crossing. How can we do that? Write the sample covariance matrix as $S$ and a covariance matrix corresponding to a perturbation of the sample $S^*$. (The perturbation could come from the bootstrap, or, conceivably, some other source). The problem now is that the eigensolutions of $S$ and $S^*$, computed by some algorithm, not necessarily are "the same", so we cannot pair of eigenvalues from $S$ with eigenvalues from $S^*$. ¿What does this mean? Write $t \mapsto t S + (1-t) S^* = S_t$. To be able to "pair of" eigenvalues we need an eigensolution of $S_t$ that depends continuously on $t$! The usual algorithms do not give us that. So, my question is: Anybody knows about such algorithms? any other related ideas? papers? Thank you!

share|improve this question
Is it possible to get a link to the paper or at least its abstract? – Michael Chernick Oct 2 '12 at 19:43
When you say that PCA breaks down, what exactly does it do that it is not suppose to do? In my mind what it is suppose to do is identify through the eigenvectors the linear combinations of the original variables that provide maximum explanation of the variance in the sample data. This is based on the sample covariance matrix. Are you saying that the result is sensitive to small perturbations in the multivariate samples? – Michael Chernick Oct 2 '12 at 19:49
Breakdown is used in the referred paper for the following: If eigenvalues are close, so crossings are probable, then the interior product between $v$ and $v_s$, where $v$ is tthe (model) eigenvector corresponding to largest (model) eigenvalue, and the corresponding sample eigenvector, comes close to zero. So we are misidentifying the signal. Yes, I am saying the result is sensitive to small perturbations, and the sensitivity depends on the eigenvalue spacings. – kjetil b halvorsen Oct 2 '12 at 20:16
reference for paper:… – kjetil b halvorsen Oct 2 '12 at 20:19
Thanks for the clarification. it helps to know what you are driving at even though i don't think I can answer the question. The idea of using the bootstrap always perks my interest. – Michael Chernick Oct 2 '12 at 20:31

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.