7
$\begingroup$

I performed two principal components analyses: in R and in SPSS - using the same dataset and the same variables. I got the same results - at least to some point. The eigenvalues are the same (I used the correlation matrix in both cases, no rotation), but the loadings are different.

I decided to take a look at some plots, so below you can see two plots of those R and SPSS PCA loadings. The first plot presents a little bit changed loadings of PCA in R. Namely, these are the opposite signs of R loadings. The second plot presents the original loading of PCA in SPSS. These plots are the same, but I can't figure out why they different in numbers.

Does R or SPSS do any implicit loadings transformation?

enter image description here

Improve this question
$\endgroup$
8
  • $\begingroup$ PCAs (not using varimax rotation) are calculated using SVDs. R uses the LAPACK linear algebra library. I don't know what SPSS uses, and the devs are not obliged to divulge since SPSS is proprietary. Intuitively, it seems reasonable that different correct implementations might be faced with this problem. So you are not quite right when you say that they are different results. They are identical results, leading to identical inference, just using differing results. You can change the signs of loadings ad liberandum. $\endgroup$
    – AdamO
    Jun 27, 2014 at 15:28
  • 1
    $\begingroup$ Is the first squared eigenvalue roughly $6$? $\endgroup$
    – whuber ♦
    Jun 27, 2014 at 15:45
  • $\begingroup$ @AdamO: ad libitum? $\endgroup$
    – Nick Cox
    Jun 27, 2014 at 16:31
  • 1
    $\begingroup$ @AdamO That's a fair translation of ad libitum, which I suggested. I think ad liberandum has quite different overtones, as shown by its Christian liturgical uses. $\endgroup$
    – Nick Cox
    Jun 27, 2014 at 16:49
  • 1
    $\begingroup$ One reason of the discrepansy may be that - I suspect - what SPSS calles eigenvectors, R calls loadings. At the end of my answer under the link, you'll find links to examples of full PCA results; use them, to see. $\endgroup$
    – ttnphns
    Jun 27, 2014 at 21:16

3 Answers 3

Reset to default
5
$\begingroup$

The difference is in how R and SPSS interpret the word "loading". Loadings in PCA should be defined as eigenvectors of the covariance matrix scaled by the square roots of the respective eigenvalues. Please see e.g. my answer here for motivation:

This is the definition followed by SPSS. However, what R (unfortunately) calls "loadings" are non-scaled eigenvectors of the covariance matrix. Therefore, your two plots should differ in scaling by a square root of the first eigenvalue. As the scaling factor seems to be $\approx 2.5$, the first eigenvalue should be approximately equal to $2.5^2=6.25$, as @whuber hinted in the comments above.

Improve this answer
$\endgroup$
3
$\begingroup$

The simple answer, by a R-guy.

About the sign-shifting; each loading-vector have an "evil twin vector" pointing in 180 degrees opposite direction in feature space. The loading vectors yield scores of opposite signs. Similar to (-1)(+1) =(+1)(-1), the solutions are identical.

Also the length of the loading vector does not really matter, as long it will be reflected by the scores. X = loadings x scores + e , and loadings x scores = X(fit) Thus loadings(R) x scores(R) = loadings(SPSS) x scores(SPSS)

The two solutions are identical

Improve this answer
$\endgroup$
3
  • $\begingroup$ it.. did't work. Before I posted my question, I tried to solve it by myself - centering and scaling. Hee are some R-commands, that I used: pca.ABC=princomp(my.base[,34:63], cor=T, center=T) pca.ABC=princomp(my.base[,34:63], cor=T, scale=T) pca.ABC=princomp(my.base[,34:63], cor=T, scale=T, center=T) pca.ABC=princomp(my.base[,34:63], cor=T, scale=T, center=F) $\endgroup$
    – Lil'Lobster
    Jun 27, 2014 at 15:12
  • $\begingroup$ I changed my first answer :) $\endgroup$
    – Soren Havelund Welling
    Jun 27, 2014 at 15:15
  • $\begingroup$ The length of the loading vector actually does matter, in a sense that there is an accepted definition (followed by SPSS, but not followed by R). Please see my answer. $\endgroup$
    – amoeba
    Dec 18, 2014 at 10:50
0
$\begingroup$

Full details on the SPSS PCA algorithm would be found in the Algorithms doc included with the product. But it appears that the results are actually identical up to an arbitrary linear transformation.

Improve this answer
$\endgroup$
1
  • $\begingroup$ This linear transformation is not arbitrary! See my answer. $\endgroup$
    – amoeba
    Dec 18, 2014 at 10:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.