Questions tagged [latent-semantic-analysis]

LSA stands for Latent Semantic Analysis, a natural language processing technique which involves analysing the relationships between documents and terms they contain by producing a set of related concepts.

Filter by
Sorted by
Tagged with
0 votes
0 answers
59 views

LSA and cosine similarity approximation with large matrix

This question is the non-r related version of this one I have a tdm with 16k x 350k dimension, for which I am trying to get the document-document similarity (cosine type). With RSpectra I have found a ...
user avatar
0 votes
0 answers
12 views

How can I compute a similarity between a term and a passage using LSA?

I am looking here: http://www.scholarpedia.org/article/Latent_semantic_analysis and I am especially interested in the sentence: "Since both passages and terms are represented as vectors, it is ...
user avatar
0 votes
0 answers
18 views

How to intepret negative coefficients in LSI model?

I am using Gensim 4 to train a LSI model. I tried to print some topic, and this is one of the result: ...
user avatar
2 votes
0 answers
653 views

What are the current approaches to topic modelling (i.e. better than LSA, LDA, LSI)?

I am looking into methods for topic modeling with the purpose of keyword generation. Given a corpus consisting of multiple documents, I would like to get a list of semantically relevant and ...
user avatar
  • 173
1 vote
0 answers
20 views

Meaning of $P(d)$ in pLSA

Given the following equation: $$ {\displaystyle P(w,d)=P(d)\sum _{c}P(c|d)P(w|c)} $$ How $P(d)$ is calculated? I suggest that's relative length of the document with respect to corpus: $$ P(d)=\frac{\...
user avatar
2 votes
0 answers
161 views

Interpretation of negative values in the results of LSA

How can I interpret the results of LSA? From the following table, I can understand that documents 0 and 1 (dealing with cats) fall into Topic 2. Document 4, which talks about both dog and cat, falls ...
user avatar
  • 121
3 votes
1 answer
269 views

Why is non-centered SVD accepted in LSA

In Latent Semantic Analysis (LSA) , we apply SVD to a term-document matrix $A$, then choose to ignore all but $k$ largest singular values. The term-document matrix is not centered, or normalised, ...
user avatar
  • 61
1 vote
1 answer
3k views

Which features to include for Truncated SVD?

I have a dataset of ~31000 8k-filings (ad-hoc announcements from companies listed on the stock exchange). Every document consists of a string (the actual filing, stemmed and stopwords removed) and a ...
user avatar
1 vote
1 answer
699 views

Scikit-learn dimension selection LSA

I am using decomposition.SVD on a TF-IDF features and I would like to know how I can improve my dimension selection? I know that scikit-learn advice to use 100 for LSA but I would like to be sure ...
user avatar
  • 115
0 votes
1 answer
392 views

Can pLSA model generate topic distribution of unseen documents?

I refer to the Wikipedia and other tutorials on topic modeling, which said although PLSA is a generative model of the documents in the collection it is estimated on, it is not a generative model ...
user avatar
  • 157
0 votes
1 answer
114 views

How Many Documents are Required for Latent Semantic Indexing?

How many documents are generally required to produce good results for latent semantic indexing? By good results I mean relevant results are given for queries
user avatar
  • 1,485
1 vote
0 answers
378 views

How to choose an optimal dimension reduction factor in LSA processing

I'm performing a K-Means clustering on a 400.000 text dataset. After eliminating useless chars and removing stopwords, I get a dictionnary size of around 42000 words. So before doing the clustering, ...
user avatar
2 votes
0 answers
395 views

Supervised semantic analysis

Dimensional reduction and semantic vectorization techniques like LSA, pLSA, LDA and Random Indexing do not take advantage of semantic labeled data like Explicit Semantic Analysis (ESA). I am looking ...
user avatar
  • 61
1 vote
1 answer
998 views

Latent Semantic Analysis: stop words and link words

On many tutorials about how to implement LSA, I see that stop words such as "and" are removed. I understand that we might find them in almost all kind of texts, but the repetition of link words in a ...
user avatar
1 vote
0 answers
35 views

Question about word/discourse similarity analysis

Question about a investigative journalistic proof involving LDA and prior along with general probability distribution model for an unusual type of problem. Basically I have an email I wrote to a ...
user avatar
  • 11
2 votes
0 answers
248 views

LSA projections of documents and terms

I am trying to understand how Latent Semantic Analysis works, reading demonstrations based on singular value decomposition. Let's denote $X$ a $D \times W$ document-term matrix. The $D$ rows of $X$ ...
user avatar
  • 41
1 vote
1 answer
582 views

Interpretation of LSI/LSA when reducing the number of documents

Usually LSI/LSA is done on a TermFrequency matrix (each row a document, each column a term) to reduce the dimensionality along the terms dimension (i.e. there are too many words). In that way we would ...
user avatar
  • 163
1 vote
0 answers
360 views

LDA or pLSA for short documents?

I'd like to classify short documents, from a predefined set of words. What algorithm would you suggest, LDA or pLSA ? My use case I have a list of users, and for each user a list of the pages she ...
user avatar
  • 1,711
5 votes
2 answers
3k views

Latent Dirichlet Allocation vs. pLSA

In the original LDA paper it is stated that: The parameters for a k-topic pLSI model are k multinomial distributions of size V and M mixtures over the k hidden topics. This gives kV +kM parameters ...
user avatar
  • 241
4 votes
1 answer
933 views

Latent Semantic Indexing and Data Centering

In PCA it's common to center the data, i.e. preprocess the data matrix such that the columns have zero mean. PCA can be done via SVD, but in this case the data matrix also has to be mean-centered. If ...
user avatar
1 vote
0 answers
2k views

Which SVD matrix do we use for cosine similarity

In Latent Semantic Analysis, we get 3 matrices from the singular value decomposition (SVD), but I am confused - which matrix do we use for cosine similarity?
user avatar
8 votes
0 answers
6k views

Is it ok to get negative Cosine Similarity using LSA? [closed]

I am getting negative cosine similarity value between two documents in Latent Semantic analysis. How should it be treated?
user avatar
3 votes
1 answer
4k views

How are the clustering algorithms using the concept of Latent Semantic Analysis?

I have come across Latent Semantic Analysis, but I could not understand it. Can Latent Semantic Analysis be used by humans in clustering of the data-sets? For convenience let us consider the datasets ...
user avatar
  • 51
10 votes
3 answers
23k views

K-means on cosine similarities vs. Euclidean distance (LSA)

I am using latent semantic analysis to represent a corpus of documents in lower dimensional space. I want to cluster these documents into two groups using k-means. Several years ago, I did this using ...
user avatar
  • 3,555
2 votes
1 answer
2k views

Different results for Singular Value Decomposition (SVD) using different tools

I am currently implementing Latent Semantic Analysis in Java using the EJML library for the preliminary Singular Value Decomposition (SVD). I am testing my code against the original term frequency ...
user avatar
  • 173
3 votes
1 answer
1k views

Latent semantic classification

How can I create a training data set for document classification using LSA? I have created a term-to-document matrix and have class labels also. I don't know whether to add these class labels in a ...
user avatar
  • 31
1 vote
1 answer
24 views

single categorical DV, single countinuous IV

Background: I have a set of student answers to some questions and also their scores, whether the answer is correct or not (1 - not correct, 2 - somewhat correct, 3 - correct). I also have for each ...
user avatar
1 vote
1 answer
1k views

How can I interpret the results of LSA?

I implemented LSA (Latent Semantic Analysis) on MATLAB. I have a $D\times N$ term-document matrix, where $D$: # of words, $N$: # of docs. I did low-rank approximation using SVD, and got $$X_k = U_k \...
user avatar
0 votes
1 answer
877 views

how to identify salient words after LSA?

I used Latent Semantic Analysis (LSA) to extract latent topics (i.e., the polynomials coefficient1*word1 + coefficient2*word2 + ...) from a certain corpus. I know that the larger the (absolute value ...
user avatar
  • 135
3 votes
0 answers
206 views

Application of LSA/LSI; Is it common to include the use of an edit distance?

I have been using Latent Semantic Analysis (LSA) or Latent Semantic Indexing (LSI) to identify whether different email addresses belong to the same individual by matching on names used for each email ...
user avatar
  • 31
2 votes
2 answers
2k views

Clustering of documents that are very different in number of words

I have a corpus of 643 documents with different sizes and my goal is to cluster them according their topics and label each cluster with semantic name for its main topic. I have tired different ...
user avatar
4 votes
1 answer
4k views

Computing document similarity in latent semantic analysis

I have a question regarding Latent Semantic Analysis - after performing SVD decomposition of term-document matrix and choosing some number of dimensions, I get the set of new document vectors. Now, ...
user avatar
  • 1,209
1 vote
0 answers
185 views

Probabilistic latent semantic analysis - objective function

In pLSA we are trying to maximize a likelihood function. The likelihood function can be given as: $L = \Pi_i^N \Pi_j^M P(d_i,w_j)^{n(d_i,w_j)}$ Where, M is the number of words and N is the number of ...
user avatar
  • 1,099
28 votes
3 answers
20k views

LSA vs. PCA (document clustering)

I'm investigation various techniques used in document clustering and I would like to clear some doubts concerning PCA (principal component analysis) and LSA (latent semantic analysis). First thing - ...
user avatar
  • 1,209
3 votes
2 answers
236 views

Is LSA and topic clustering easier in European languages similar to English?

I was watching a talk on latent semantic analysis and the speaker described experience applying LSA and REALLY messy data. He concluded that it demonstrated the difficulty of disambiguation of meaning,...
user avatar
3 votes
1 answer
2k views

Using LSA for dimension reduction on test data

I am looking to explore using LSA as a dimension reduction technique for some textual data. To be specific I would like to take a matrix of M documents and N n-grams (ie: variables) and then create a ...
user avatar
  • 141
1 vote
1 answer
396 views

pLSA using tempered EM

In an article by Hofmann pdf, he proposes: initialize $β$ to one, run until convergence, then rescale $β$ by a factor $η<1$, run again until convergence, and iterate this until changing $β$ ...
user avatar
10 votes
1 answer
2k views

A parellel between LSA and pLSA

In the original paper of pLSA the author, Thomas Hoffman, draw a parallel between pLSA and LSA data structures that I would like to discuss with you. Background: Taking inspiration the Information ...
user avatar
  • 728
1 vote
1 answer
386 views

Derivation of M-step for pLSA

I was looking at section 6 of these notes and trying to understand the derivation of the M-step at the top of page 10. I understood the derivation for the model without background, but I do not ...
user avatar
  • 113
2 votes
0 answers
299 views

Initialization in pLSA

I'm using pLSA (Probabilistic Latent Semantic Analysis). I'm trying to estimate the best value for some parameters with a process of k-fold cross-validation. I have noticed that the model fitting is ...
user avatar
  • 728
6 votes
1 answer
7k views

How to cluster LDA/LSI topics generated by gensim?

I'm an enthusiastic single developer working on a small start-up idea. I reduced a corpus of mine to an LSA/LDA vector space using gensim. Now I have a bunch of ...
user avatar
  • 61
13 votes
4 answers
3k views

Fast alternatives to the EM algorithm

Are there any speedy alternatives to the EM algorithm for learning models with latent variables (especially pLSA)? I'm okay with sacrificing precision in favor of speed.
user avatar
  • 728
2 votes
1 answer
124 views

Is there a sequential version of probabilistic latent semantic analysis?

Does someone know if it exists some way to do online learning with pLSA? The model training is really time consuming, so it is not feasible to rebuild it after every changes in the data.
user avatar
  • 728
4 votes
0 answers
172 views

Finding similarity between a reference and few working documents

I have to find the similarity between a reference document and a set of documents in a repository . Here is my method : ...
user avatar
3 votes
1 answer
6k views

Can LSA be used for document similarity?

I have to find the similarity between two documents. The two documents are simple text documents and i have to report a score. I was using cosine similarity initially. But I was told that LSA is a ...
user avatar
9 votes
1 answer
6k views

When to choose PCA vs. LSA/LSI

Question: Are there any general guidelines with respect to the input data characteristics, that can be used to decide between applying PCA versus LSA/LSI? Brief summary of PCA vs. LSA/LSI: ...
user avatar
  • 221
4 votes
1 answer
1k views

pLSA - Probabilistic Latent Semantic Analysis, how to choose topic number?

I am learning about pLSA (Probabilistic Latent Semantic Analysis) right now, in the hopes of being able to apply it to biomolecular annotation prediction. I have a very simple question: How do you ...
user avatar
5 votes
2 answers
937 views

What is a "tempered EM algorithm"?

In the paper of Probabilistic Latent Semantic Analysis by Hofmann, the author fits the model for document $\times$ word matrix through EM Algorithm in section 3. I was able to follow the derivation ...
user avatar
  • 4,067
3 votes
1 answer
1k views

Derivation of E step in EM algorithm

While im going through the derivation of E step in EM algorithm for pLSA, i came across the following derivation at this page. Could anyone explain me how the following step is derived. $\sum_z q(z) ...
user avatar
  • 4,067
6 votes
2 answers
815 views

Deriving mathematical model of pLSA

After knowing how LSA works, I went on continue reading on pLSA but couldn't really make sense of the mathematical formula. This is what I get from wikipedia (other academic papers/tutorial show ...
user avatar
  • 187