In PCA it's common to center the data, i.e. preprocess the data matrix such that the columns have zero mean. PCA can be done via SVD, but in this case the data matrix also has to be mean-centered. If we don't center it, the found principal directions will not make sense.
But in LSA/LSI when we apply SVD to the term-document matrix, we do not center it - at least I haven't found any paper saying that we should.
For example, here's a figure from (1):
Although this illustration serves quite a different purpose (to show that NMF is better than LSI), we see data is not centered here. Why is that? Why we do not need centering for LSI?
The closest explanation that I found in (2) is that centering does not preserve the cosine similarity between documents.
But without centering the principal directions of LSI seem to be meaningless: in PCA we want to find directions of most variance, and if we don't center the data, the found directions will correspond to something else. So without centering it will be some projection, but not necessarily a good one in the PCA sense.
So why do we do LSI this way?
References:
- Xu, Wei, Xin Liu, and Yihong Gong. "Document clustering based on non-negative matrix factorization." 2003.
- Korenius, Tuomo, Jorma Laurikkala, and Martti Juhola. "On principal component analysis, cosine and Euclidean measures in information retrieval." 2007.
that centering does not preserve the cosine similarity between documents
True. You'll get correlation coefficient then.without centering the principal directions of LSI are meaningless
Why necessarily so? don't be hurry to say it. $\endgroup$ – ttnphns May 18 '15 at 21:00directions will correspond to something else
Directions of maximal sum-of-squares. PCA or similar SVD-based methods always maximizes the explained SS. If the data cloud is centered then SS is called scatter and SS/n [or n-1] is called variance. See also. $\endgroup$ – ttnphns May 19 '15 at 14:40