These terms get thrown around together a lot, but I would like to know what you think the differences are, if any.
Thanks
Tell me more
×
Cross Validated is a question and answer site for
statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
|
|
||||
|
|
|
LSA and LSI are mostly used synonymously, with the information retrieval community usually referring to it as LSI. LSA/LSI uses SVD to decompose the term-document matrix A into a term-concept matrix U, a singular value matrix S, and a concept-document matrix V in the form: A = USV'. The wikipedia page has a detailed description of latent semantic indexing. |
|||
|
|
|
Notably while LSA and LSI use SVD to do their magic, there is a computationally and conceptually simpler method called HAL (Hyperspace Analogue to Language) that sifts through text keeping track of preceding and subsequent contexts. Vectors are extracted from these (often weighted) co-occurrence matrices and specific words are selected to index the semantic space. In many ways I'm given to understand it performs as well as LSA without requiring the mathematically/conceptually complex step of SVD. See Lund & Burgess, 1996 for details. |
|||||||||
|
|
NMF and SVD are both matrix factorization algorithms. Wikipedia has some relevant information on NMF. SVD and PCA are intimately related. For starters, PCA is simply the eigendecomposition of the correlation. SVD is a generalization of eigendecomposition to non-square matrices. The singular values are the square root of the eigenvalues of the matrix multiplied by its transpose (making it square, and amenable to eigendecomposition). Furthermore, if the matrix is normal ($A^*A=A A^*$), the singular values are simply the absolute values of the eigenvalues. In any case, the singular values are non-negative, and losing the sign of the eigenvalues is the price you pay for being able to work with non-square matrices. The other responders have covered LSI/LSA... |
||||
|
|
