4
$\begingroup$

I understand the sign of the eigen vectors / PCA rotations can be positive or negative (see here or here).

But I am curious why the following two approaches yield different results, from numerical method perspective?

> d = iris[,1:2]
> pca = prcomp(d)
> r = eigen(cov(d))
> r$vectors
            [,1]        [,2]
[1,] -0.99640834 -0.08467831
[2,]  0.08467831 -0.99640834
> pca$rotation
                     PC1        PC2
Sepal.Length  0.99640834 0.08467831
Sepal.Width  -0.08467831 0.99640834
Improve this question
$\endgroup$
3
  • 4
    $\begingroup$ I'm not sure this is an exact duplicate - although it is closely related. (OP here already knows that eigenvectors are only defined up to a change in sign, wants to know why different computational approaches give different answers ...) $\endgroup$ – Ben Bolker Sep 7 '16 at 22:49
  • 1
    $\begingroup$ @Ben You might be right, but this issue has come up so many times I'm sure there's a duplicate around somewhere. In the meantime because I was certain it would be closed as a dup eventually, I wanted to close it as a dup of something. (+1 for your answer, BTW: it adds a welcome computing perspective to the other answers I recall seeing.) $\endgroup$ – whuber♦ Sep 7 '16 at 23:11
  • $\begingroup$ Note that a PCA function (prcomp in this instance) might be adjusting the sign after the eigendecompsition. Often, a PCA function will change the sign of eigenverctors so that the sum in each eigenvector is positive. Just for convenience. $\endgroup$ – ttnphns Sep 8 '16 at 14:33
8
$\begingroup$

Looking at the code, stats:::prcomp.default uses singular value decomposition at its core, rather than eigendecomposition of the variance-covariance matrix.

In general, the computed signs of eigenvectors can be very sensitive to small computational differences: according to Brian Ripley on R-help in 2003,

using different compilers on the same machine and the same version of R may give different signs for the eigenvectors. The moral is, don't rely on the signs of eigenvectors! (This is on the help page.)

That is, it's not just prcomp (SVD) vs. eigen; the results could differ for eigen across operating systems, compilers, possibly even floating-point hardware ...

Improve this answer
$\endgroup$
6
  • 2
    $\begingroup$ Could you tell me why? The results could differ for eigen across operating systems, compilers, possibly even floating-point hardware? where is the "randomness" in the different systems? $\endgroup$ – Haitao Du Sep 8 '16 at 13:50
  • $\begingroup$ @hxd1011 I do not think it is random. The same software on the same OS on the same computer will give the same result but changing one of them may give a different result. $\endgroup$ – mdewey Sep 8 '16 at 14:00
  • $\begingroup$ @mdewey thanks, but why different system will be different? is ieee 754 is implemented differently cross OS? like TCP IP stack? $\endgroup$ – Haitao Du Sep 8 '16 at 14:05
  • 1
    $\begingroup$ @Ben, The claim in your answer is extremely important. One thing is when different functions/programs give different signs. Another thing (potentially threatening) is that on different OS or computers the results will be sign-different. Can you support with evidence your claim? $\endgroup$ – ttnphns Sep 8 '16 at 14:19
  • 1
    $\begingroup$ Well, you could take Brian Ripley's word for it :-) ... I don't have the time to run around trying eigen() on different OS/computers at the moment, but I'm sure it's true. The point here is that there are certain aspects of floating-point math that are not stable across platforms - knowing which ones is important. $\endgroup$ – Ben Bolker Sep 8 '16 at 14:22
0
$\begingroup$

I know this is an old topic, but it's happened to me recently and I have found a solution that seems to work. I guess is due the internals of prcomp and eigen as pointed out in a previous comment, but if you want eigen to produce the same eigenvectors as prcomp it seems using the flag symmetric = FALSE does the trick:

> d = iris[,1:2]
> pca = prcomp(d)
> r = eigen(cov(d), symmetric = FALSE)
> r$vectors
            [,1]       [,2]
[1,]  0.99640834 0.08467831
[2,] -0.08467831 0.99640834
> pca$rotation
                     PC1        PC2
Sepal.Length  0.99640834 0.08467831
Sepal.Width  -0.08467831 0.99640834
>

I hope this is useful for someone else.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.