2
$\begingroup$

I'm trying to run PCA on sample covariance matrices of various sizes (ranging between 20 x 20 to 4000 x 4000). Assume the data follows a joint multivariate normal distribution.

While derivations are great, I'm asking from an applied perspective. Bonus points for easy-to-understand papers and packages in R that help with the following:

  1. Is there a way to test the statistical significance of the eigenvalues? Or assign a probability of a given eigenvalue occurring?

  2. How does the relative magnitude of the eigenvalues change as we scale up the size of the covariance matrix? E.g. Let's say the first 3 principal components explain ~80% of the variance for a 20x20 matrix (not sure if this is true in practice). If we were to scale this up to a 4000x4000 matrix, would we still expect to see ~80% of the variance explained by the first 3 PCs?

Assume the population covariance matrix is not diagonal and is of full rank. Thanks!

$\endgroup$

closed as unclear what you're asking by whuber May 13 at 17:31

Please clarify your specific problem or add additional details to highlight exactly what you need. As it's currently written, it’s hard to tell exactly what you're asking. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.

  • 3
    $\begingroup$ For "statistical significance of eigenvalues" to have any meaning, you need to specify a null hypothesis. What do you have in mind for that, especially in light of your desire to assume a non-diagonal covariance matrix? Is your question (2) about data or about the model? $\endgroup$ – whuber Aug 11 '16 at 19:07
  • 3
    $\begingroup$ So how familiar are you with results in statweb.stanford.edu/~imj/WEBLIST/2001/… ? $\endgroup$ – StasK Aug 11 '16 at 19:10
  • $\begingroup$ @StasK I glanced through the paper but the results in the paper all assume the identity as the data covariance matrix .... please correct me if I'm wrong $\endgroup$ – feynmanisabro Aug 11 '16 at 20:02
  • 1
    $\begingroup$ Trying to clarify the same two questions asked by @whuber: (1) So you have a particular fixed population cov matrix C that you are willing to assume for your null? (2) This is not clear until you specify how your C is supposed to change "as the matrix grows". $\endgroup$ – amoeba Aug 11 '16 at 22:34
  • 1
    $\begingroup$ (1) Okay, this is clear now. (2) If all data are uncorrelated and N(0,1) then the population covariance matrix is equal to identity and the first PC explains 1/n of total variance. Clearly this goes to 0 as n grows. However, if your cov matrix is different, then it can be anything. You can choose C such that PC1 explains 100% of the variance with any n. This remains an ill-posed question. $\endgroup$ – amoeba Aug 13 '16 at 13:04
0
$\begingroup$

You may want to look the Wikipedia entry "Random Matrix" . Among other things It provides pointers to the work of Marcenko-Pastur which may be of interest if I understand your question correctly.

$\endgroup$
  • $\begingroup$ As far as I understand, this distribution requires the data matrix entries to be i.i.d. i.e. a diagonal correlation matrix $\endgroup$ – feynmanisabro Aug 12 '16 at 0:45

Not the answer you're looking for? Browse other questions tagged or ask your own question.