The Stack Overflow podcast is back! Listen to an interview with our new CEO.

Questions tagged [lsa]

LSA stands for Latent Semantic Analysis, a natural language processing technique which involves analysing the relationships between documents and terms they contain by producing a set of related concepts.

47 questions
Filter by
Sorted by
Tagged with
22 views

Latent Semantic Indexing coordinates vs PCA scores

For PCA: Let $X$ be a mean-centered matrix having as columns observations and as rows features. Via SVD, we have $X = U \Sigma V^\top$. Then PCA scores are $U^\top X = \Sigma V^\top$. For LSA/LSI: ...
11 views

How to compare documents in LSA?

In wikipedia: https://en.wikipedia.org/wiki/Latent_semantic_analysis it is mentioned in Derivation section "....You can now do the following: See how related documents j and q are in the low-...
22 views

Meaning of matrix U in LSA?

LSA uses SVD algorithm i.e. the terms-documents matrix $=U \Sigma V^T$ What is the exact meaning of matrix U here: I know that it is a rotation but rotation of what? Is it a concepts space? If yes ...
364 views

Why can't K-means be used on LDA output

I started working on a topic definition task and my initial approach was as follows: Use LDA (Latent Dirichlet Allocation) to obtain the initial topic distribution for each of my documents. Then use ...
2k views

Which features to include for Truncated SVD?

I have a dataset of ~31000 8k-filings (ad-hoc announcements from companies listed on the stock exchange). Every document consists of a string (the actual filing, stemmed and stopwords removed) and a ...
487 views

Scikit-learn dimension selection LSA

I am using decomposition.SVD on a TF-IDF features and I would like to know how I can improve my dimension selection? I know that scikit-learn advice to use 100 for LSA but I would like to be sure ...
279 views

Can pLSA model generate topic distribution of unseen documents?

I refer to the Wikipedia and other tutorials on topic modeling, which said although PLSA is a generative model of the documents in the collection it is estimated on, it is not a generative model ...
89 views

How Many Documents are Required for Latent Semantic Indexing?

How many documents are generally required to produce good results for latent semantic indexing? By good results I mean relevant results are given for queries
342 views

How to choose an optimal dimension reduction factor in LSA processing

I'm performing a K-Means clustering on a 400.000 text dataset. After eliminating useless chars and removing stopwords, I get a dictionnary size of around 42000 words. So before doing the clustering, ...
375 views

Supervised semantic analysis

Dimensional reduction and semantic vectorization techniques like LSA, pLSA, LDA and Random Indexing do not take advantage of semantic labeled data like Explicit Semantic Analysis (ESA). I am looking ...
794 views

Latent Semantic Analysis: stop words and link words

On many tutorials about how to implement LSA, I see that stop words such as "and" are removed. I understand that we might find them in almost all kind of texts, but the repetition of link words in a ...
34 views

Question about word/discourse similarity analysis

Question about a investigative journalistic proof involving LDA and prior along with general probability distribution model for an unusual type of problem. Basically I have an email I wrote to a ...
181 views

LSA projections of documents and terms

I am trying to understand how Latent Semantic Analysis works, reading demonstrations based on singular value decomposition. Let's denote $X$ a $D \times W$ document-term matrix. The $D$ rows of $X$ ...
440 views

Interpretation of LSI/LSA when reducing the number of documents

Usually LSI/LSA is done on a TermFrequency matrix (each row a document, each column a term) to reduce the dimensionality along the terms dimension (i.e. there are too many words). In that way we would ...
302 views

LDA or pLSA for short documents?

I'd like to classify short documents, from a predefined set of words. What algorithm would you suggest, LDA or pLSA ? My use case I have a list of users, and for each user a list of the pages she ...
2k views

Latent Dirichlet Allocation vs. pLSA

In the original LDA paper it is stated that: The parameters for a k-topic pLSI model are k multinomial distributions of size V and M mixtures over the k hidden topics. This gives kV +kM parameters ...
672 views

Latent Semantic Indexing and Data Centering

In PCA it's common to center the data, i.e. preprocess the data matrix such that the columns have zero mean. PCA can be done via SVD, but in this case the data matrix also has to be mean-centered. If ...
1k views

Which SVD matrix do we use for cosine similarity

In Latent Semantic Analysis, we get 3 matrices from the singular value decomposition (SVD), but I am confused - which matrix do we use for cosine similarity?
4k views

Is it ok to get negative Cosine Similarity using LSA?

I am getting negative cosine similarity value between two documents in Latent Semantic analysis. How should it be treated?
3k views

How are the clustering algorithms using the concept of Latent Semantic Analysis?

I have come across Latent Semantic Analysis, but I could not understand it. Can Latent Semantic Analysis be used by humans in clustering of the data-sets? For convenience let us consider the datasets ...
18k views

K-means on cosine similarities vs. Euclidean distance (LSA)

I am using latent semantic analysis to represent a corpus of documents in lower dimensional space. I want to cluster these documents into two groups using k-means. Several years ago, I did this using ...
1k views

Different results for Singular Value Decomposition (SVD) using different tools

I am currently implementing Latent Semantic Analysis in Java using the EJML library for the preliminary Singular Value Decomposition (SVD). I am testing my code against the original term frequency ...
1k views

Latent semantic classification

How can I create a training data set for document classification using LSA? I have created a term-to-document matrix and have class labels also. I don't know whether to add these class labels in a ...
21 views

single categorical DV, single countinuous IV

Background: I have a set of student answers to some questions and also their scores, whether the answer is correct or not (1 - not correct, 2 - somewhat correct, 3 - correct). I also have for each ...
704 views

How can I interpret the results of LSA?

I implemented LSA (Latent Semantic Analysis) on MATLAB. I have a $D\times N$ term-document matrix, where $D$: # of words, $N$: # of docs. I did low-rank approximation using SVD, and got X_k = U_k \...
705 views

how to identify salient words after LSA?

I used Latent Semantic Analysis (LSA) to extract latent topics (i.e., the polynomials coefficient1*word1 + coefficient2*word2 + ...) from a certain corpus. I know that the larger the (absolute value ...
155 views

Application of LSA/LSI; Is it common to include the use of an edit distance?

I have been using Latent Semantic Analysis (LSA) or Latent Semantic Indexing (LSI) to identify whether different email addresses belong to the same individual by matching on names used for each email ...
1k views

Clustering of documents that are very different in number of words

I have a corpus of 643 documents with different sizes and my goal is to cluster them according their topics and label each cluster with semantic name for its main topic. I have tired different ...
3k views

Computing document similarity in latent semantic analysis

I have a question regarding Latent Semantic Analysis - after performing SVD decomposition of term-document matrix and choosing some number of dimensions, I get the set of new document vectors. Now, ...
179 views

Probabilistic latent semantic analysis - objective function

In pLSA we are trying to maximize a likelihood function. The likelihood function can be given as: $L = \Pi_i^N \Pi_j^M P(d_i,w_j)^{n(d_i,w_j)}$ Where, M is the number of words and N is the number of ...
14k views

LSA vs. PCA (document clustering)

I'm investigation various techniques used in document clustering and I would like to clear some doubts concerning PCA (principal component analysis) and LSA (latent semantic analysis). First thing - ...
211 views

Is LSA and topic clustering easier in European languages similar to English?

I was watching a talk on latent semantic analysis and the speaker described experience applying LSA and REALLY messy data. He concluded that it demonstrated the difficulty of disambiguation of meaning,...
1k views

Using LSA for dimension reduction on test data

I am looking to explore using LSA as a dimension reduction technique for some textual data. To be specific I would like to take a matrix of M documents and N n-grams (ie: variables) and then create a ...
382 views

pLSA using tempered EM

In an article by Hofmann pdf, he proposes: initialize $β$ to one, run until convergence, then rescale $β$ by a factor $η<1$, run again until convergence, and iterate this until changing $β$ ...
2k views

A parellel between LSA and pLSA

In the original paper of pLSA the author, Thomas Hoffman, draw a parallel between pLSA and LSA data structures that I would like to discuss with you. Background: Taking inspiration the Information ...
335 views

Derivation of M-step for pLSA

I was looking at section 6 of these notes and trying to understand the derivation of the M-step at the top of page 10. I understood the derivation for the model without background, but I do not ...
243 views

Initialization in pLSA

I'm using pLSA (Probabilistic Latent Semantic Analysis). I'm trying to estimate the best value for some parameters with a process of k-fold cross-validation. I have noticed that the model fitting is ...
6k views

How to cluster LDA/LSI topics generated by gensim?

I'm an enthusiastic single developer working on a small start-up idea. I reduced a corpus of mine to an LSA/LDA vector space using gensim. Now I have a bunch of ...
2k views

Fast alternatives to the EM algorithm

Are there any speedy alternatives to the EM algorithm for learning models with latent variables (especially pLSA)? I'm okay with sacrificing precision in favor of speed.
114 views

Is there a sequential version of probabilistic latent semantic analysis?

Does someone know if it exists some way to do online learning with pLSA? The model training is really time consuming, so it is not feasible to rebuild it after every changes in the data.
165 views

Finding similarity between a reference and few working documents

I have to find the similarity between a reference document and a set of documents in a repository . Here is my method : ...
4k views

Can LSA be used for document similarity?

I have to find the similarity between two documents. The two documents are simple text documents and i have to report a score. I was using cosine similarity initially. But I was told that LSA is a ...
5k views

When to choose PCA vs. LSA/LSI

Question: Are there any general guidelines with respect to the input data characteristics, that can be used to decide between applying PCA versus LSA/LSI? Brief summary of PCA vs. LSA/LSI: ...
974 views

pLSA - Probabilistic Latent Semantic Analysis, how to choose topic number?

I am learning about pLSA (Probabilistic Latent Semantic Analysis) right now, in the hopes of being able to apply it to biomolecular annotation prediction. I have a very simple question: How do you ...
772 views

What is a “tempered EM algorithm”?

In the paper of Probabilistic Latent Semantic Analysis by Hofmann, the author fits the model for document $\times$ word matrix through EM Algorithm in section 3. I was able to follow the derivation ...