6
$\begingroup$

I have a couple of questions regarding differences in loading values when using prcomp and principal (from the psych) package to perform PCA

When conducting PCA using prcomp:

pca_results <- prcomp(df, center = TRUE, scale. = TRUE)
pca_results$rotation[,1:5]

I get the following loadings:

              PC1          PC2           PC3           PC4           PC5
q3   -0.016809164  0.134292686 -0.1757822345  1.108893e-01 -0.1319508350
q5    0.050866015 -0.161877460  0.0892043331  2.767157e-02  0.1474154691
q8   -0.008870246 -0.015767530  0.0115132365  1.618538e-01  0.2722705733

When using principal from the psych package:

pca_fit <- principal(df, nfactors = 5, rotate = "none")
pca_fit$loadings

I get these loading values instead:

     PC1    PC2    PC3    PC4    PC5   
q3          -0.335 -0.369  0.207  0.211
q5   -0.149  0.403  0.187        -0.235
q8                         0.301 -0.435

My first question: why is there a difference in the loading values between the 2 methods? Which is correct? I've also tried prcomp with center and scale set to FALSE but the numbers still don't match

My second question: what does the missingness in the principal loadings indicate? Is it some threshold of loading value, below which nothing is shown?

$\endgroup$
  • 1
    $\begingroup$ Please do not mix "loadings" with "eigenvectors". prcomp improperly calls eigenvectors "loadings". Search this site for PCA loadings eigenvectors, to learn the difference. $\endgroup$ – ttnphns Feb 18 '17 at 6:23
4
$\begingroup$

As I can see from your data - the difference is only in scaling. For instance, PC3 is scaled with $psych = 2.099188243053083 * prcomp$, some scaled with negative number. So, both algorithms are correct as principal direction doesn't change with positive scaling, and negative scaling encodes the same space.

In order to see the whole picture - you can check out eigenvalues (from sdev - a squared root of eigenvalue):

pca_results <- prcomp(df, center = TRUE, scale. = TRUE)
> pca_results$rotation[,1:4]
                    PC1         PC2        PC3        PC4
Sepal.Length  0.5210659 -0.37741762  0.7195664  0.2612863
Sepal.Width  -0.2693474 -0.92329566 -0.2443818 -0.1235096
Petal.Length  0.5804131 -0.02449161 -0.1421264 -0.8014492
Petal.Width   0.5648565 -0.06694199 -0.6342727  0.5235971

> pca_results$sdev
[1] 1.7083611 0.9560494 0.3830886 0.1439265

Compare it with psych's loadings ("eigenvalues" are called SS-loadings here):

> pca_fit <- principal(df, nfactors = 4, rotate = "none")
> pca_fit$loadings

Loadings:
             PC1    PC2    PC3    PC4   
Sepal.Length  0.890  0.361 -0.276       
Sepal.Width  -0.460  0.883              
Petal.Length  0.992                0.115
Petal.Width   0.965         0.243       

                 PC1   PC2   PC3   PC4
SS loadings    2.918 0.914 0.147 0.021
Proportion Var 0.730 0.229 0.037 0.005
Cumulative Var 0.730 0.958 0.995 1.000

You can see that they're same (after squaring). So the algorithms output same factors, they just represent principal directions differently: eigenvectors in prcomp (unit length), loadings in psych (non-unit length).

So, the only "problem" with psych is that its vector doesn't have unit length. The prcomp vector has length 1, for instance for PC1:

prcomp: 0.5210659 * 0.5210659 + 0.2693474 * 0.2693474 + 0.5804131 * 0.5804131 + 0.5648565 * 0.5648565 = 0.99999992627343

psych: 0.890 * 0.890 + 0.460 * 0.460 + 0.992 * 0.992 + 0.965 * 0.965 = 2.918989

P.S. Yes, psych package doesn't show values less than epsilon, that's why there's some empty cells there. Edit: As @William Revelle pointed out "the reason the small loadings are dropped is that the loadings object is of class "loadings" and R print drops values less than .3. Unclass the loadings with (unclass(pca_fit$loadings) and then print them."


As a matter of fact, loadings in psych are not unit eigenvectors because it uses them for factor rotation. That's why loadings are eigenvectors scaled by the square roots of the respective eigenvalues (even if you specify no rotation). See https://stats.stackexchange.com/a/137003/137805

$\endgroup$
  • $\begingroup$ ok thanks. Finally, is epsilon some sort of special value in PCA? Why is that chosen as the threshold? $\endgroup$ – Simon Feb 18 '17 at 4:15
  • $\begingroup$ no, I think psych is using it just to improve readability; coz psych is less "scientific" I'd say :). Another instance is that eigenvectors should have unit length, psych's vectors don't have it - you have to normalize them (rescale), prcomp gives you vectors with unit length by default - that's why you see difference in results. $\endgroup$ – dk14 Feb 18 '17 at 4:20
  • 1
    $\begingroup$ It is better to avoid calling eigenvectors (unit-normalized principal directions) "loadings" altogether. There is no "load" in them. stats.stackexchange.com/a/143949/3277 $\endgroup$ – ttnphns Feb 18 '17 at 6:42
  • $\begingroup$ @ttnphns Thanks, I corrected the answer to highlight "principal directions" term $\endgroup$ – dk14 Feb 18 '17 at 7:01
  • 1
    $\begingroup$ Actually, the reason the small loadings are dropped is that the loadings object is of class "loadings" and R print drops values less than .3 Unclass the loadings (unclass(pca_fit$loadings) and then print them. Better yet is to actually print the pca results which shows the loadings without deleting the small values. $\endgroup$ – William Revelle Feb 9 '18 at 22:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.