4
$\begingroup$

I understand the sign of the eigen vectors / PCA rotations can be positive or negative (see here or here).

But I am curious why the following two approaches yield different results, from numerical method perspective?

> d = iris[,1:2]
> pca = prcomp(d)
> r = eigen(cov(d))
> r$vectors
            [,1]        [,2]
[1,] -0.99640834 -0.08467831
[2,]  0.08467831 -0.99640834
> pca$rotation
                     PC1        PC2
Sepal.Length  0.99640834 0.08467831
Sepal.Width  -0.08467831 0.99640834
Improve this question
$\endgroup$
3
  • 4
    $\begingroup$ I'm not sure this is an exact duplicate - although it is closely related. (OP here already knows that eigenvectors are only defined up to a change in sign, wants to know why different computational approaches give different answers ...) $\endgroup$
    – Ben Bolker
    Sep 7, 2016 at 22:49
  • 1
    $\begingroup$ @Ben You might be right, but this issue has come up so many times I'm sure there's a duplicate around somewhere. In the meantime because I was certain it would be closed as a dup eventually, I wanted to close it as a dup of something. (+1 for your answer, BTW: it adds a welcome computing perspective to the other answers I recall seeing.) $\endgroup$
    – whuber ♦
    Sep 7, 2016 at 23:11
  • $\begingroup$ Note that a PCA function (prcomp in this instance) might be adjusting the sign after the eigendecompsition. Often, a PCA function will change the sign of eigenverctors so that the sum in each eigenvector is positive. Just for convenience. $\endgroup$
    – ttnphns
    Sep 8, 2016 at 14:33

2 Answers 2

Reset to default
8
$\begingroup$

Looking at the code, stats:::prcomp.default uses singular value decomposition at its core, rather than eigendecomposition of the variance-covariance matrix.

In general, the computed signs of eigenvectors can be very sensitive to small computational differences: according to Brian Ripley on R-help in 2003,

using different compilers on the same machine and the same version of R may give different signs for the eigenvectors. The moral is, don't rely on the signs of eigenvectors! (This is on the help page.)

That is, it's not just prcomp (SVD) vs. eigen; the results could differ for eigen across operating systems, compilers, possibly even floating-point hardware ...

Improve this answer
$\endgroup$
6
  • 2
    $\begingroup$ Could you tell me why? The results could differ for eigen across operating systems, compilers, possibly even floating-point hardware? where is the "randomness" in the different systems? $\endgroup$
    – Haitao Du
    Sep 8, 2016 at 13:50
  • $\begingroup$ @hxd1011 I do not think it is random. The same software on the same OS on the same computer will give the same result but changing one of them may give a different result. $\endgroup$
    – mdewey
    Sep 8, 2016 at 14:00
  • $\begingroup$ @mdewey thanks, but why different system will be different? is ieee 754 is implemented differently cross OS? like TCP IP stack? $\endgroup$
    – Haitao Du
    Sep 8, 2016 at 14:05
  • 1
    $\begingroup$ @Ben, The claim in your answer is extremely important. One thing is when different functions/programs give different signs. Another thing (potentially threatening) is that on different OS or computers the results will be sign-different. Can you support with evidence your claim? $\endgroup$
    – ttnphns
    Sep 8, 2016 at 14:19
  • 1
    $\begingroup$ Well, you could take Brian Ripley's word for it :-) ... I don't have the time to run around trying eigen() on different OS/computers at the moment, but I'm sure it's true. The point here is that there are certain aspects of floating-point math that are not stable across platforms - knowing which ones is important. $\endgroup$
    – Ben Bolker
    Sep 8, 2016 at 14:22
0
$\begingroup$

I know this is an old topic, but it's happened to me recently and I have found a solution that seems to work. I guess is due the internals of prcomp and eigen as pointed out in a previous comment, but if you want eigen to produce the same eigenvectors as prcomp it seems using the flag symmetric = FALSE does the trick:

> d = iris[,1:2]
> pca = prcomp(d)
> r = eigen(cov(d), symmetric = FALSE)
> r$vectors
            [,1]       [,2]
[1,]  0.99640834 0.08467831
[2,] -0.08467831 0.99640834
> pca$rotation
                     PC1        PC2
Sepal.Length  0.99640834 0.08467831
Sepal.Width  -0.08467831 0.99640834
>

I hope this is useful for someone else.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.