I am trying to understand how Latent Semantic Analysis works, reading demonstrations based on singular value decomposition.

Let's denote $X$ a $D \times W$ document-term matrix. The $D$ rows of $X$ represent documents, and the $W$ columns represent words. Using SVD, we can write $X = U \Sigma V^T$.

Furthermore, we define $\tilde{X}_k$ as the truncated rank $k$ approximation of $X$. Keeping only the columns and rows in $\Sigma$ corresponding to the largest elements on its diagonal, and the associated columns/rows of $U$ and $V^T$, we have $\tilde{X} = U_k \Sigma_k V^T_k$.

In this context, I would like to ask two questions (related, I think):

  1. Assume we have a new document, represented by row-vector $d$. We would like to get a rank $k$ approximation of $d$ by applying the same mapping that transformed rows in $X$ into rows in $\tilde{X}$ and get $\hat{d}$. I read that $\hat{d} = \Sigma^{-1}_k U_k^T d$, but I can't find a full, formal mathematical demonstration behind this equation. Any clue?

  2. Is there any way to write the columns (terms vectors) of $\tilde{X}_k$ as a linear combination of the columns in $X$? With Python's gensim library, it seems indeed possible to write any topic (so any new semantic vector) as a linear combination of words.

Any help would be greatly appreciated. Also, any good reference on the topic would be most welcome!


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.