Questions tagged [latent-semantic-analysis]
LSA stands for Latent Semantic Analysis, a natural language processing technique which involves analysing the relationships between documents and terms they contain by producing a set of related concepts.
54
questions
0
votes
0
answers
19
views
Interpretation of LSA results
I perform an LSA with textmodels_lsa of the quanteda package in R, but I have little idea about interpreting the results.
A minimal example taken from here
...
- 1
3
votes
1
answer
197
views
How to compare the semantic similarity of text generated by large language models (GPT-3, BLOOM etc) to reference text?
Large language models, such as GPT-3, BLOOM etc, can generate open-ended text. Say I want to prompt these models to answer a question. How can I compare the semantic similarity of the answer it ...
- 31
0
votes
0
answers
20
views
Should I do documents transformation at once or a pair at a time for auto grading with cosine similarity?
I'm developing auto grading essay that compares the similarity between the answer key and student answer with cosine similarity. This one is written in php. Let's say in a course there are 30 - 100 ...
3
votes
1
answer
91
views
What is the best method to find the sentences that answer a given question in a document?
Let's say I have a long document (let's say 10 pages) and I have a question about the content of the document. An example could be:
Document: History of Flowers
Question: What type of flowers have ...
- 33
0
votes
0
answers
23
views
Is word2vec or LSA a better model for disambiguating word similarity
I have an assignment coming up that asks of me to use either word2vec or LSA to disambiguate similarity between words. Which model do you think would perform better and why?
I know that the two models ...
1
vote
0
answers
30
views
Why do we call word embeddings "vectors" ? Shouldn't we call them word coordinates?
Word embeddings are often referred to as vectors. For instance, "Word embedding models are dense vector representations of words learned from a corpus of natural language" (Rosado, 2020). ...
- 552
1
vote
0
answers
57
views
Input for LSI, LDA and HDP?
It seems that the Latent Semantic Indexing (LSI or Latent Semantic Analysis, LSA) can take as input information frequency data (such as TfIdf-weighted space), e.g., according to this documentation ...
- 31
2
votes
0
answers
818
views
What are the current approaches to topic modelling (i.e. better than LSA, LDA, LSI)?
I am looking into methods for topic modeling with the purpose of keyword generation. Given a corpus consisting of multiple documents, I would like to get a list of semantically relevant and ...
- 173
1
vote
0
answers
28
views
Meaning of $P(d)$ in pLSA
Given the following equation:
$$
{\displaystyle P(w,d)=P(d)\sum _{c}P(c|d)P(w|c)}
$$
How $P(d)$ is calculated? I suggest that's relative length of the document with respect to corpus:
$$
P(d)=\frac{\...
- 11
2
votes
0
answers
251
views
Interpretation of negative values in the results of LSA
How can I interpret the results of LSA?
From the following table, I can understand that documents 0 and 1 (dealing with cats) fall into Topic 2.
Document 4, which talks about both dog and cat, falls ...
- 131
3
votes
1
answer
394
views
Why is non-centered SVD accepted in LSA
In Latent Semantic Analysis (LSA) , we apply SVD to a term-document matrix $A$, then choose to ignore all but $k$ largest singular values.
The term-document matrix is not centered, or normalised, ...
- 61
1
vote
1
answer
3k
views
Which features to include for Truncated SVD?
I have a dataset of ~31000 8k-filings (ad-hoc announcements from companies listed on the stock exchange). Every document consists of a string (the actual filing, stemmed and stopwords removed) and a ...
- 31
1
vote
1
answer
756
views
Scikit-learn dimension selection LSA
I am using decomposition.SVD on a TF-IDF features and I would like to know how I can improve my dimension selection?
I know that scikit-learn advice to use 100 for LSA but I would like to be sure ...
- 115
1
vote
1
answer
432
views
Can pLSA model generate topic distribution of unseen documents?
I refer to the Wikipedia and other tutorials on topic modeling, which said
although PLSA is a generative model of the documents in the collection it is estimated on, it is not a generative model ...
- 207
1
vote
1
answer
142
views
How Many Documents are Required for Latent Semantic Indexing?
How many documents are generally required to produce good results for latent semantic indexing? By good results I mean relevant results are given for queries
- 1,855
1
vote
0
answers
419
views
How to choose an optimal dimension reduction factor in LSA processing
I'm performing a K-Means clustering on a 400.000 text dataset. After eliminating useless chars and removing stopwords, I get a dictionnary size of around 42000 words.
So before doing the clustering, ...
- 203
2
votes
0
answers
403
views
Supervised semantic analysis
Dimensional reduction and semantic vectorization techniques like LSA, pLSA, LDA and Random Indexing do not take advantage of semantic labeled data like Explicit Semantic Analysis (ESA). I am looking ...
- 71
1
vote
1
answer
1k
views
Latent Semantic Analysis: stop words and link words
On many tutorials about how to implement LSA, I see that stop words such as "and" are removed. I understand that we might find them in almost all kind of texts, but the repetition of link words in a ...
- 203
1
vote
0
answers
35
views
Question about word/discourse similarity analysis
Question about a investigative journalistic proof involving LDA and prior along with general probability distribution model for an unusual type of problem.
Basically I have an email I wrote to a ...
- 11
2
votes
0
answers
272
views
LSA projections of documents and terms
I am trying to understand how Latent Semantic Analysis works, reading demonstrations based on singular value decomposition.
Let's denote $X$ a $D \times W$ document-term matrix. The $D$ rows of $X$ ...
- 41
1
vote
1
answer
653
views
Interpretation of LSI/LSA when reducing the number of documents
Usually LSI/LSA is done on a TermFrequency matrix (each row a document, each column a term) to reduce the dimensionality along the terms dimension (i.e. there are too many words). In that way we would ...
- 163
1
vote
0
answers
378
views
LDA or pLSA for short documents?
I'd like to classify short documents, from a predefined set of words.
What algorithm would you suggest, LDA or pLSA ?
My use case
I have a list of users, and for each user a list of the pages she ...
- 1,751
5
votes
2
answers
3k
views
Latent Dirichlet Allocation vs. pLSA
In the original LDA paper it is stated that:
The parameters for a k-topic pLSI model are k multinomial distributions of size V and M mixtures over the k hidden topics. This gives kV +kM parameters ...
- 251
4
votes
1
answer
1k
views
Latent Semantic Indexing and Data Centering
In PCA it's common to center the data, i.e. preprocess the data matrix such that the columns have zero mean. PCA can be done via SVD, but in this case the data matrix also has to be mean-centered. If ...
- 8,617
1
vote
0
answers
2k
views
Which SVD matrix do we use for cosine similarity
In Latent Semantic Analysis, we get 3 matrices from the singular value decomposition (SVD), but I am confused - which matrix do we use for cosine similarity?
- 11
8
votes
0
answers
7k
views
Is it ok to get negative Cosine Similarity using LSA? [closed]
I am getting negative cosine similarity value between two documents in Latent Semantic analysis. How should it be treated?
- 81
3
votes
1
answer
5k
views
How are the clustering algorithms using the concept of Latent Semantic Analysis?
I have come across Latent Semantic Analysis, but I could not understand it.
Can Latent Semantic Analysis be used by humans in clustering of the data-sets? For convenience let us consider the datasets ...
- 51
10
votes
3
answers
24k
views
K-means on cosine similarities vs. Euclidean distance (LSA)
I am using latent semantic analysis to represent a corpus of documents in lower dimensional space. I want to cluster these documents into two groups using k-means.
Several years ago, I did this using ...
- 3,605
2
votes
1
answer
2k
views
Different results for Singular Value Decomposition (SVD) using different tools
I am currently implementing Latent Semantic Analysis in Java using the EJML library for the preliminary Singular Value Decomposition (SVD). I am testing my code against the original term frequency ...
- 173
3
votes
1
answer
1k
views
Latent semantic classification
How can I create a training data set for document classification using LSA? I have created a term-to-document matrix and have class labels also. I don't know whether to add these class labels in a ...
- 31
1
vote
1
answer
24
views
single categorical DV, single countinuous IV
Background:
I have a set of student answers to some questions and also their scores, whether the answer is correct or not (1 - not correct, 2 - somewhat correct, 3 - correct).
I also have for each ...
1
vote
1
answer
2k
views
How can I interpret the results of LSA?
I implemented LSA (Latent Semantic Analysis) on MATLAB.
I have a $D\times N$ term-document matrix, where $D$: # of words, $N$: # of docs.
I did low-rank approximation using SVD, and got $$X_k = U_k \...
- 11
0
votes
1
answer
966
views
how to identify salient words after LSA?
I used Latent Semantic Analysis (LSA) to extract latent topics (i.e., the polynomials coefficient1*word1 + coefficient2*word2 + ...) from a certain corpus. I know that the larger the (absolute value ...
- 135
3
votes
0
answers
216
views
Application of LSA/LSI; Is it common to include the use of an edit distance?
I have been using Latent Semantic Analysis (LSA) or Latent Semantic Indexing (LSI) to identify whether different email addresses belong to the same individual by matching on names used for each email ...
- 31
2
votes
2
answers
2k
views
Clustering of documents that are very different in number of words
I have a corpus of 643 documents with different sizes and my goal is to cluster them according their topics and label each cluster with semantic name for its main topic.
I have tired different ...
5
votes
1
answer
4k
views
Computing document similarity in latent semantic analysis
I have a question regarding Latent Semantic Analysis - after performing SVD decomposition of term-document matrix and choosing some number of dimensions, I get the set of new document vectors.
Now, ...
- 1,279
1
vote
0
answers
196
views
Probabilistic latent semantic analysis - objective function
In pLSA we are trying to maximize a likelihood function.
The likelihood function can be given as:
$L = \Pi_i^N \Pi_j^M P(d_i,w_j)^{n(d_i,w_j)}$
Where, M is the number of words and N is the number of ...
- 1,109
29
votes
3
answers
20k
views
LSA vs. PCA (document clustering)
I'm investigation various techniques used in document clustering and I would like to clear some doubts concerning PCA (principal component analysis) and LSA (latent semantic analysis).
First thing - ...
- 1,279
3
votes
2
answers
240
views
Is LSA and topic clustering easier in European languages similar to English?
I was watching a talk on latent semantic analysis and the speaker described experience applying LSA and REALLY messy data. He concluded that it demonstrated the difficulty of disambiguation of meaning,...
- 207
3
votes
1
answer
2k
views
Using LSA for dimension reduction on test data
I am looking to explore using LSA as a dimension reduction technique for some textual data. To be specific I would like to take a matrix of M documents and N n-grams (ie: variables) and then create a ...
- 141
1
vote
1
answer
411
views
pLSA using tempered EM
In an article by Hofmann pdf, he proposes:
initialize $β$ to one, run until convergence, then rescale $β$ by a
factor $η<1$, run again until convergence, and iterate this until
changing $β$ ...
10
votes
1
answer
2k
views
A parellel between LSA and pLSA
In the original paper of pLSA the author, Thomas Hoffman, draw a parallel between pLSA and LSA data structures that I would like to discuss with you.
Background:
Taking inspiration the Information ...
- 756
1
vote
1
answer
411
views
Derivation of M-step for pLSA
I was looking at section 6 of these notes and trying to understand the derivation of the M-step at the top of page 10. I understood the derivation for the model without background, but I do not ...
- 113
2
votes
0
answers
309
views
Initialization in pLSA
I'm using pLSA (Probabilistic Latent Semantic Analysis). I'm trying to estimate the best value for some parameters with a process of k-fold cross-validation.
I have noticed that the model fitting is ...
- 756
6
votes
1
answer
7k
views
How to cluster LDA/LSI topics generated by gensim?
I'm an enthusiastic single developer working on a small start-up idea. I reduced a corpus of mine to an LSA/LDA vector space using gensim. Now I have a bunch of ...
- 61
13
votes
4
answers
3k
views
Fast alternatives to the EM algorithm
Are there any speedy alternatives to the EM algorithm for learning models with latent variables (especially pLSA)? I'm okay with sacrificing precision in favor of speed.
- 756
2
votes
1
answer
128
views
Is there a sequential version of probabilistic latent semantic analysis?
Does someone know if it exists some way to do online learning with pLSA? The model training is really time consuming, so it is not feasible to rebuild it after every changes in the data.
- 756
4
votes
0
answers
173
views
Finding similarity between a reference and few working documents
I have to find the similarity between a reference document and a set of documents in a repository .
Here is my method :
...
- 71
3
votes
1
answer
6k
views
Can LSA be used for document similarity?
I have to find the similarity between two documents. The two documents are simple text documents and i have to report a score. I was using cosine similarity initially. But I was told that LSA is a ...
- 71
9
votes
1
answer
6k
views
When to choose PCA vs. LSA/LSI
Question:
Are there any general guidelines with respect to the input data characteristics, that can be used to decide between applying PCA versus LSA/LSI?
Brief summary of PCA vs. LSA/LSI:
...
- 221