I have following example of principal component analysis using first 4 variables of iris data set (code in R):

> res = prcomp(iris[1:4])
> res
Standard deviations:
[1] 2.0562689 0.4926162 0.2796596 0.1543862

                     PC1         PC2         PC3        PC4
Sepal.Length  0.36138659 -0.65658877  0.58202985  0.3154872
Sepal.Width  -0.08452251 -0.73016143 -0.59791083 -0.3197231
Petal.Length  0.85667061  0.17337266 -0.07623608 -0.4798390
Petal.Width   0.35828920  0.07548102 -0.54583143  0.7536574

It appears that Sepal.Width has a very small contribution to PC1. How do I know if it is a significant contribution?

Is there any significance test for this? Similarly, I want to determine significance for all values in above matrix.

Also, is there any package in R that does it?


This is not (yet) and answer, only a comment but too long for the box

I do not really know how to determine such significance; but out of couriosity I did a bootstrap-procedure: from a replication of the original data to a pseudo-population of $N=19200$ I draw $t=1000$ randomsamples of $n=150$ (each row of the dataset could occur at most $128$ times).
From each of this $t=1000$ experiments I computed the pca-solutions and stored the first pc only in a list. From this 1000 instances of first pc's I got the following statistics for their loadings:

PrC[1]:  Mean      Min       Max    Stddev  SE_mean  lb(95%)     mean  ub(95%) 
   S.L    0.362    0.314    0.412    0.015    0.000    0.361    0.362    0.362
   S.W   -0.085   -0.131   -0.023    0.017    0.001   -0.086   -0.085   -0.083
   P.L    0.856    0.841    0.869    0.004    0.000    0.856    0.856    0.857
   P.W    0.358    0.334    0.382    0.008    0.000    0.358    0.358    0.359

The 95% confidence interval for the item S.Width was -0.085 .. - 0.083 and this shows that this value seems to be from zero not by the pure random-effect of the sampling. (Similarly narrow appear all 95% confidence intervals for the other loadings)
After that it's clear I need more clarification what it means for a loading to "contribute significantly" - significance derived from what expectance? (But that's what I do not yet understand, I'm competely illiterate yet with the question of significance-estimation for covariances and for loadings in a factormodel, so this all might be of no help at all here)

[Update 2]
Here is a picture which shows the location of the Iris-items in the coordinates of the first 2 principal components, evaluated by the Monte-Carlo-experiment ("population": $N=128 \cdot 150=19200$, "sample": $n=150$, number-of-samples: $s=1000$)

Picture 1: (using covariance-matrix, loadings from eigenvectors as done in the OP's question)
From the picture I'd say, that the small loading of Sepal.Width of -0.141 on pc1 is a reliable (different from zero, however small) estimate of the loading in the "population" (because the whole cloud is separated from the y-axis)

Using the standard interpretation of PCA (based on correlations, using scaled eigenvectors) the picture looks a bit different, but still with very little disturbances of the loadings of the items.
The statistics are as in the following:

PrC[1]  Mean      Min       Max    Stddev  SE_mean  lb(95%)     mean  ub(95%) 
   S.L    0.891    0.840    0.937    0.015    0.000    0.890    0.891    0.892
   S.W   -0.459   -0.705   -0.159    0.081    0.003   -0.465   -0.459   -0.454
   P.L    0.991    0.987    0.994    0.001    0.000    0.991    0.991    0.991
   P.W    0.965    0.946    0.980    0.005    0.000    0.965    0.965    0.965

Picture 2: (using correlation-matrix, principal components taken in the standard method)

[Update 1] Just for my own couriosity I made a set of plots of the empirical loadings-matrices when samples are drawn from a known population. That's somehow bootstrapping, and I've not yet seen similar images. I took as population a set of 1000 normal random distributed cases with a certain factorial structure. Then I draw 256 random samples from the population with n=40 and did the same components-analysis/rotation for each of that 256 samples. To compare and to see, how the accuracy of the estimation improves I took the same number of samples, but now each sample with n=160. See the comparision at http://go.helms-net.de/stat/sse/StabilityofPC

| cite | improve this answer | |
  • $\begingroup$ Thanks for your answer. What exact code did you use for above bootstrap procedure? Did you encounter sign issue, since loadings sometimes come with reverse signs on repeated sampling? $\endgroup$ – rnso Jun 26 '15 at 13:30
  • $\begingroup$ I concatened the original dataset several times until N=19200 was reached. Then by a randomgenerator which was restricted to the range 1..19200 (with guaranteed frequency of 1 for each random value if called less than 19200 times) I simply took the first 150 numbers and used them as index into the population-dataset. This made one sample. Evaluate, store the first prcomp and repeat until 1000 samples and thus 1000 first prcomps were collected. Then did simple descriptive statistics on the list. Sign-issue? Hmm, I don't know what you mean? $\endgroup$ – Gottfried Helms Jun 26 '15 at 14:38
  • $\begingroup$ See stackoverflow.com/questions/31057192/… $\endgroup$ – rnso Jun 26 '15 at 15:15
  • 1
    $\begingroup$ @rnso: What if not just the signs flip but also the axes of the PCA when choosing different bootstrap samples? (Imagine e.g. the case where the original variables are quite orthogonal). I am asking of pure curiosity. $\endgroup$ – Michael M Apr 18 '16 at 14:12
  • 1
    $\begingroup$ The figures you added seem to be very close to the ones I produced answering your follow-up question. That's good. $\endgroup$ – amoeba Apr 18 '16 at 14:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.