Difference in PCA loadings between R and SPSS

I performed two principal components analyses: in R and in SPSS - using the same dataset and the same variables. I got the same results - at least to some point. The eigenvalues are the same (I used the correlation matrix in both cases, no rotation), but the loadings are different.

I decided to take a look at some plots, so below you can see two plots of those R and SPSS PCA loadings. The first plot presents a little bit changed loadings of PCA in R. Namely, these are the opposite signs of R loadings. The second plot presents the original loading of PCA in SPSS. These plots are the same, but I can't figure out why they different in numbers.

Does R or SPSS do any implicit loadings transformation?

• PCAs (not using varimax rotation) are calculated using SVDs. R uses the LAPACK linear algebra library. I don't know what SPSS uses, and the devs are not obliged to divulge since SPSS is proprietary. Intuitively, it seems reasonable that different correct implementations might be faced with this problem. So you are not quite right when you say that they are different results. They are identical results, leading to identical inference, just using differing results. You can change the signs of loadings ad liberandum. – AdamO Jun 27 '14 at 15:28
• Is the first squared eigenvalue roughly $6$? – whuber Jun 27 '14 at 15:45
• @AdamO: ad libitum? – Nick Cox Jun 27 '14 at 16:31
• @AdamO That's a fair translation of ad libitum, which I suggested. I think ad liberandum has quite different overtones, as shown by its Christian liturgical uses. – Nick Cox Jun 27 '14 at 16:49
• One reason of the discrepansy may be that - I suspect - what SPSS calles eigenvectors, R calls loadings. At the end of my answer under the link, you'll find links to examples of full PCA results; use them, to see. – ttnphns Jun 27 '14 at 21:16

The difference is in how R and SPSS interpret the word "loading". Loadings in PCA should be defined as eigenvectors of the covariance matrix scaled by the square roots of the respective eigenvalues. Please see e.g. my answer here for motivation:

This is the definition followed by SPSS. However, what R (unfortunately) calls "loadings" are non-scaled eigenvectors of the covariance matrix. Therefore, your two plots should differ in scaling by a square root of the first eigenvalue. As the scaling factor seems to be $\approx 2.5$, the first eigenvalue should be approximately equal to $2.5^2=6.25$, as @whuber hinted in the comments above.

The simple answer, by a R-guy.

About the sign-shifting; each loading-vector have an "evil twin vector" pointing in 180 degrees opposite direction in feature space. The loading vectors yield scores of opposite signs. Similar to (-1)(+1) =(+1)(-1), the solutions are identical.

Also the length of the loading vector does not really matter, as long it will be reflected by the scores. X = loadings x scores + e , and loadings x scores = X(fit) Thus loadings(R) x scores(R) = loadings(SPSS) x scores(SPSS)

The two solutions are identical

• it.. did't work. Before I posted my question, I tried to solve it by myself - centering and scaling. Hee are some R-commands, that I used: pca.ABC=princomp(my.base[,34:63], cor=T, center=T) pca.ABC=princomp(my.base[,34:63], cor=T, scale=T) pca.ABC=princomp(my.base[,34:63], cor=T, scale=T, center=T) pca.ABC=princomp(my.base[,34:63], cor=T, scale=T, center=F) – Lil'Lobster Jun 27 '14 at 15:12
• I changed my first answer :) – Soren Havelund Welling Jun 27 '14 at 15:15
• The length of the loading vector actually does matter, in a sense that there is an accepted definition (followed by SPSS, but not followed by R). Please see my answer. – amoeba says Reinstate Monica Dec 18 '14 at 10:50

Full details on the SPSS PCA algorithm would be found in the Algorithms doc included with the product. But it appears that the results are actually identical up to an arbitrary linear transformation.

• This linear transformation is not arbitrary! See my answer. – amoeba says Reinstate Monica Dec 18 '14 at 10:49