2
$\begingroup$

As far as I know, singular value decomposition (SVD) and eigendecomposition give the same result for symmetric square matrices. But when I check the results in R, that's not what I see. Please see below R code (I set the random seed for reproducibility purposes):

set.seed(111)
X = matrix(rnorm(50), nrow=5, ncol=10)
prinden = svd(X %*% t(X))
prineden = eigen(X %*% t(X))

> prinden$u
       [,1]       [,2]        [,3]       [,4]        [,5]
[1,] -0.3812420  0.1714440  0.89298930 -0.1352677  0.09764363
[2,] -0.3086450 -0.1785482  0.05935694  0.9181008 -0.16256214
[3,]  0.2125363  0.8719446 -0.06555398  0.3011303  0.31553776
[4,] -0.6109540  0.4041138 -0.30752420 -0.2079534 -0.57062372
[5,] -0.5839147 -0.1230124 -0.31650973 -0.0697814  0.73407341

> prineden$vectors
       [,1]       [,2]        [,3]       [,4]        [,5]
[1,] -0.3812420 -0.1714440  0.89298930 -0.1352677  0.09764363
[2,] -0.3086450  0.1785482  0.05935694  0.9181008 -0.16256214
[3,]  0.2125363 -0.8719446 -0.06555398  0.3011303  0.31553776
[4,] -0.6109540 -0.4041138 -0.30752420 -0.2079534 -0.57062372
[5,] -0.5839147  0.1230124 -0.31650973 -0.0697814  0.73407341

The 2nd columns of prinden$u and prineden$vectors are negative of each other, while other columns are the same. How come is this possible? What am I missing?

$\endgroup$
  • 2
    $\begingroup$ Sign does not matter. It depends on the specific implementation of the function. If you negate the second column the results are identical. So these are actually two equivalent decomposition outputs. $\endgroup$ – ttnphns May 24 '15 at 4:55
  • $\begingroup$ This is very closely related to stats.stackexchange.com/questions/88880. $\endgroup$ – amoeba May 24 '15 at 7:02
1
$\begingroup$

The decomposition is not unique.

First of all, any reflection is a proper decomposition, too.

Secondly, if you have identical eigenvalues, any rotation within these axes is a proper decomposition, too.

In particular, any identity ("eye") + rotation matrix is a decomposition of any other identity ("eye") or rotation matrix (because any vector is a unit vector of them).

Such solutions do arise in practise when you use e.g. power iterations to find only the first few eigenvectors (numerically, not analytically, obviously. Reality isn't exact.)

$\endgroup$
  • $\begingroup$ I don't think "eye matrix" is a mathematical term. You are probably referring to the eye() function in Matlab, but what it returns is called an identity matrix. $\endgroup$ – amoeba May 24 '15 at 10:38
  • $\begingroup$ eye matrix may be less ambiguous. Identity can happen on so many levels. Personally I would have preferred the term identity, but have the impression some people find eye more understandable for stats users. $\endgroup$ – Anony-Mousse May 25 '15 at 0:24
2
$\begingroup$

Also, if your square matrix is indefinite (and thus has negative eigenvalues), the SVD won't capture that since it always returns nonnegative singular values.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.