2
$\begingroup$

I am currently implementing Latent Semantic Analysis in Java using the EJML library for the preliminary Singular Value Decomposition (SVD). I am testing my code against the original term frequency matrix provided in an oldish introductory paper by Landauer et al.: http://lsa.colorado.edu/papers/dp1.LSAintro.pdf

Strangely enough, I am getting slightly different results in the U matrix of the SVD compared to the results in the paper. Columns 2, 6, 8 and 9 are the negative of the results in the paper. Even stranger, I am getting yet another different results (compared to both EJML and the paper) when using GNU Octave and an online tool (http://www.bluebit.gr/matrix-calculator/calculate.aspx). In this latter case, columns 1, 7, 8, and 9 are the negative of the results in the paper.

The exact values are visible here:

Results image http://postimg.org/image/c5kkrk5h5/

(note that U is called W in the paper)

This is the original term frequency matrix (tab delimited): 

1   0   0   1   0   0   0   0   0
1   0   1   0   0   0   0   0   0
1   1   0   0   0   0   0   0   0
0   1   1   0   1   0   0   0   0
0   1   1   2   0   0   0   0   0
0   1   0   0   1   0   0   0   0
0   1   0   0   1   0   0   0   0
0   0   1   1   0   0   0   0   0
0   1   0   0   0   0   0   0   1
0   0   0   0   0   1   1   1   0
0   0   0   0   0   0   1   1   1
0   0   0   0   0   0   0   1   1

It would be interesting to compare this to other tools and languages (R, Matlab ...) to see what results these would yield. So, if you have time to run the SVD in a different environment, it would be great, if you could post the results here.

Would anyone have any idea where these differences might come from?

Thanks a lot.

Cheers,

Martin

Improve this question
$\endgroup$

1 Answer 1

Reset to default
4
$\begingroup$

SVD decomposition of a mtrix is in general not unique. If U and V are a decomposition of a matrix, then for any diagonal matrix with only -1 or 1 as diagonal elements, UM and M^TV will also be valid SVD decomposition. Different implementations maybe just randomly choose one or the other one. The PCA decomposition has the same issue: only the orientations but not the directions of the eigenvectors (singular vectors) are unique.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Hi James, thanks a lot for your reply. Just to clarify, "M^TV" stands for the transpose of matrix M multiplied by V, right? I am not enirely sure I understand the details, though. I don't want to take up too much your time, but if you happen to have a simple example at hand, I'd be grateful. Based on what you have written above, the fact that the data plays out so nicely in the paper appears to be a bit of co-incidence. $\endgroup$
    – Martin Wunderlich
    Commented Jul 8, 2014 at 18:41
  • 1
    $\begingroup$ Yes, M^T denotes the transpose of M. For non-square matrix or degenerated matrices you probably need to adjust the dimensions accordingly. The simplest example is when the matrix is just a 1x1 matrix, an constant (a). Then, the SVD decomposition U and V maybe (1) and (1); or (-1) and (-1). Since (1)(a)(1) == (-1)(a)(-1). $\endgroup$
    – James LI
    Commented Jul 9, 2014 at 20:00
  • $\begingroup$ Thanks a lot, James, for the additional comment. I had another thing about the problem. So, in essence the reason why the values in the columns differ in their sign and not in their values is that the columns reprent eigenvectors. And an eigenvector is an eigenvector regardless of the direction that it is pointing in. Hence it doesn't matter if the values are multiplied by -1. $\endgroup$
    – Martin Wunderlich
    Commented Jul 10, 2014 at 20:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.