2
$\begingroup$

I am trying to understand how Latent Semantic Analysis works, reading demonstrations based on singular value decomposition.

Let's denote $X$ a $D \times W$ document-term matrix. The $D$ rows of $X$ represent documents, and the $W$ columns represent words. Using SVD, we can write $X = U \Sigma V^T$.

Furthermore, we define $\tilde{X}_k$ as the truncated rank $k$ approximation of $X$. Keeping only the columns and rows in $\Sigma$ corresponding to the largest elements on its diagonal, and the associated columns/rows of $U$ and $V^T$, we have $\tilde{X} = U_k \Sigma_k V^T_k$.

In this context, I would like to ask two questions (related, I think):

  1. Assume we have a new document, represented by row-vector $d$. We would like to get a rank $k$ approximation of $d$ by applying the same mapping that transformed rows in $X$ into rows in $\tilde{X}$ and get $\hat{d}$. I read that $\hat{d} = \Sigma^{-1}_k U_k^T d$, but I can't find a full, formal mathematical demonstration behind this equation. Any clue?

  2. Is there any way to write the columns (terms vectors) of $\tilde{X}_k$ as a linear combination of the columns in $X$? With Python's gensim library, it seems indeed possible to write any topic (so any new semantic vector) as a linear combination of words.

Any help would be greatly appreciated. Also, any good reference on the topic would be most welcome!

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.