How to Use LSA Create Topics?

Just want to know the general process of creating document topics via LSA. For creating document clusters, I know first I should get SVD dimensions and then use k-means clustering on these SVD dimensions to create document clusters. For creating document topics, I know I also first create SVD dimensions. Then what to do after this? Are these SVD dimensions are our topics? Or we need to any further processing.

1 Answer

Once you take an SVD of the term/document matrix (or tf-idf or other representation $X$, with terms as rows and documents as columns), you obtain $\begin{matrix} X = U \Sigma V^T\end{matrix}$.

Each of the rows of $\Sigma V^T$ will then be representations of your documents in the latent space. We can truncate these representations to be an arbitrary number of dimensions d << D by just taking the top d elements. We define 'top' here as meaning those elements with the largest singular values in the diagonal $\Sigma$. In most SVD implementations the the rows of $V$ will already be ordered according to these singular values, so we can just take the first d dimensions.

Then we will have a representation of our documents in the lower-dimensional latent semantic space. To get document topics, you still need to do some sort of clustering, but your clustering algorithm will find working in the lower-dimensional space much easier.

• Thanks, and how to algorithmically give topic names? – Dyin Aug 7 '17 at 17:44
• I mean, that's a pretty difficult one. You could perhaps give a topic the most frequent word in documents of that topic? Or maybe better still, give it the most 'discriminative' word, which you could define as perhaps the word with the highest (occurrence rate in that topic's documents) / (average occurrence rate over all documents) – nlml Aug 7 '17 at 21:04