I've accidentally come across some papers discussing distributions of principal components of the sample covariance matrices. An example of such a paper is Johnstone, 2001, On the distribution of the largest eigenvalue in principal components analysis.

Not to mention that the distribution seems quite complicated (as expected), but even if we know this distribution -- how can we use it in our data mining process?

  • 4
    $\begingroup$ +1. Johnstone's result can be used to do a significance test: if the PC1 variance is high enough then we can reject the null hypothesis that the data are random. This is similar to what is known as "parallel analysis". In practice, people are often reluctant to rely on the normality assumption and perform some nonparametric shuffling procedure instead. $\endgroup$ – amoeba Oct 14 '15 at 9:34
  • $\begingroup$ I guess I could provide an answer illustrating what I said above, but it would require some work; let me know if this is interesting of if you had something else in mind. Also, I slightly edited your question, please check that everything is correct. $\endgroup$ – amoeba Oct 14 '15 at 12:27
  • $\begingroup$ Hi tanks for the answer -- but would you explain a bit more before working to provide a formal answer, that what do you mean by the data is "random"? $\endgroup$ – Wudanao Oct 14 '15 at 14:50
  • $\begingroup$ Also, is that the main use of the principal component's distribution? Personally though Im not sure exactly what you mean by “random", I think there are other much easier ways to test if data is random... $\endgroup$ – Wudanao Oct 14 '15 at 14:51
  • $\begingroup$ Imagine a dataset with 10 variables and 100 data points. You compute correlation matrix and perform PCA. You find out that the first PC explains 15% of the variance. Is it "significant"? What number would you expect if your 100*10 dataset was randomly generated with all variables being uncorrelated? With very large number of data points, all sample correlations would be tiny, so PC1 would explain ~1/10*100% = 10% of the variance. But with only 100 data points you will get some nonzero sample correlations due to chance variations and PC1 will explain more variance. Is 15% higher than that? $\endgroup$ – amoeba Oct 14 '15 at 15:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.