This is really a side project of mine ... while writing on a paper on something totally different! I read (part of ) the excellent paper


This shows among other things that misbehavior of PCA is connected to "crossings of eigenvalues". If you are interested in estimating the eigenvector corresponding to the largest (model) eigenvalue, then if the largest sample eigenvalue frequently comes from a smaller (model) eigenvalue, which is a "crossing", the corresponding sample eigenvector do not correspond to what we try to estimate, and PCA breaks down.

So, given a sample, it would be interesting to estimate (for example, using the bootstrap) the probability of crossing. How can we do that? Write the sample covariance matrix as $S$ and a covariance matrix corresponding to a perturbation of the sample $S^*$. (The perturbation could come from the bootstrap, or, conceivably, some other source). The problem now is that the eigensolutions of $S$ and $S^*$, computed by some algorithm, not necessarily are "the same", so we cannot pair of eigenvalues from $S$ with eigenvalues from $S^*$. ¿What does this mean? Write $t \mapsto t S + (1-t) S^* = S_t$. To be able to "pair of" eigenvalues we need an eigensolution of $S_t$ that depends continuously on $t$! The usual algorithms do not give us that. So, my question is: Anybody knows about such algorithms? any other related ideas? papers?


Since I did'nt get any answers here, I asked a related questio on Mathoverflow: https://mathoverflow.net/questions/116123/how-to-find-define-eigenvectors-as-a-continuous-function-of-matrix which got good answers.

  • $\begingroup$ Is it possible to get a link to the paper or at least its abstract? $\endgroup$ – Michael R. Chernick Oct 2 '12 at 19:43
  • $\begingroup$ When you say that PCA breaks down, what exactly does it do that it is not suppose to do? In my mind what it is suppose to do is identify through the eigenvectors the linear combinations of the original variables that provide maximum explanation of the variance in the sample data. This is based on the sample covariance matrix. Are you saying that the result is sensitive to small perturbations in the multivariate samples? $\endgroup$ – Michael R. Chernick Oct 2 '12 at 19:49
  • $\begingroup$ Breakdown is used in the referred paper for the following: If eigenvalues are close, so crossings are probable, then the interior product between $v$ and $v_s$, where $v$ is tthe (model) eigenvector corresponding to largest (model) eigenvalue, and the corresponding sample eigenvector, comes close to zero. So we are misidentifying the signal. Yes, I am saying the result is sensitive to small perturbations, and the sensitivity depends on the eigenvalue spacings. $\endgroup$ – kjetil b halvorsen Oct 2 '12 at 20:16
  • $\begingroup$ reference for paper: projecteuclid.org/… $\endgroup$ – kjetil b halvorsen Oct 2 '12 at 20:19
  • 2
    $\begingroup$ Thanks for the clarification. it helps to know what you are driving at even though i don't think I can answer the question. The idea of using the bootstrap always perks my interest. $\endgroup$ – Michael R. Chernick Oct 2 '12 at 20:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.