2
$\begingroup$

I'm trying to run PCA on sample covariance matrices of various sizes (ranging between 20 x 20 to 4000 x 4000). Assume the data follows a joint multivariate normal distribution.

While derivations are great, I'm asking from an applied perspective. Bonus points for easy-to-understand papers and packages in R that help with the following:

  1. Is there a way to test the statistical significance of the eigenvalues? Or assign a probability of a given eigenvalue occurring?

  2. How does the relative magnitude of the eigenvalues change as we scale up the size of the covariance matrix? E.g. Let's say the first 3 principal components explain ~80% of the variance for a 20x20 matrix (not sure if this is true in practice). If we were to scale this up to a 4000x4000 matrix, would we still expect to see ~80% of the variance explained by the first 3 PCs?

Assume the population covariance matrix is not diagonal and is of full rank. Thanks!

$\endgroup$
  • 3
    $\begingroup$ For "statistical significance of eigenvalues" to have any meaning, you need to specify a null hypothesis. What do you have in mind for that, especially in light of your desire to assume a non-diagonal covariance matrix? Is your question (2) about data or about the model? $\endgroup$ – whuber Aug 11 '16 at 19:07
  • 3
    $\begingroup$ So how familiar are you with results in statweb.stanford.edu/~imj/WEBLIST/2001/… ? $\endgroup$ – StasK Aug 11 '16 at 19:10
  • $\begingroup$ @StasK I glanced through the paper but the results in the paper all assume the identity as the data covariance matrix .... please correct me if I'm wrong $\endgroup$ – feynmanisabro Aug 11 '16 at 20:02
  • 1
    $\begingroup$ Trying to clarify the same two questions asked by @whuber: (1) So you have a particular fixed population cov matrix C that you are willing to assume for your null? (2) This is not clear until you specify how your C is supposed to change "as the matrix grows". $\endgroup$ – amoeba says Reinstate Monica Aug 11 '16 at 22:34
  • 1
    $\begingroup$ (1) Okay, this is clear now. (2) If all data are uncorrelated and N(0,1) then the population covariance matrix is equal to identity and the first PC explains 1/n of total variance. Clearly this goes to 0 as n grows. However, if your cov matrix is different, then it can be anything. You can choose C such that PC1 explains 100% of the variance with any n. This remains an ill-posed question. $\endgroup$ – amoeba says Reinstate Monica Aug 13 '16 at 13:04
0
$\begingroup$

You may want to look the Wikipedia entry "Random Matrix" . Among other things It provides pointers to the work of Marcenko-Pastur which may be of interest if I understand your question correctly.

$\endgroup$
  • $\begingroup$ As far as I understand, this distribution requires the data matrix entries to be i.i.d. i.e. a diagonal correlation matrix $\endgroup$ – feynmanisabro Aug 12 '16 at 0:45

Not the answer you're looking for? Browse other questions tagged or ask your own question.