All Questions
Tagged with lda or latent-dirichlet-alloc
141 questions
1
vote
1
answer
29
views
How does Gibbs sampling end up "sampling from the posterior"? [duplicate]
Let me preface this by saying I have very little experience in statistics. I did my stats courses in undergrad and most of those pertained to things like regression, not Bayesian inference.
However, ...
- 11
0
votes
1
answer
67
views
About the use of Bayes' rule for continuous valued random variables
I am currently studying the book "An introduction to statistical learning with application in Python" and I am currently at the part 4 of chapter 4 where they explain the general framework ...
- 3
1
vote
0
answers
16
views
Should I evaluate topic coherence of topic model on test set?
I'm a newbie in topic models. From my understanding, topic models like latent dirichlet allocation (LDA) (without empirical Bayes on hyperparameters) performs variational inference to approximate the ...
- 211
0
votes
0
answers
27
views
Reformulating latent dirichlet allocation
can you guys help me solve this equation and go from :
to this one :
I have tried something but my result gives me this : p(θ, φ, z| α, β) = p(θ, φ, z, w|α, β)/p(w|α, β)
the equations are taken from ...
- 1
0
votes
0
answers
43
views
LDA: sample mean of two distributions are proportional
I came up with a discussion around what does it mean that the sample mean of two distributions are proportional. This discussion started when talking about linear discriminant analysis and how this ...
- 185
1
vote
0
answers
123
views
LDA Document Topic Distribution Prediction for Unseen Document
Popular python libraries for topic modeling like gensim or sklearn allow us to predict the topic-distribution for an unseen document, but I have a few questions on what's going on under the hood. I've ...
0
votes
0
answers
74
views
Understanding Variational inference for LDA
I am trying to derive from scratch variational inference for LDA. I am following this course: https://home.cs.colorado.edu/~jbg/teaching/CSCI_5622/19a.pdf
When computing $p(Z|\Theta)$ they do the ...
- 449
0
votes
1
answer
90
views
How does Gibbs sampling work in Latent Dirichlet Allocation? It seems that only one sample is sampled from the distribution of corpus topics
I'm very new to both Gibbs sampling and LDA. Currently, I'm trying to understand the collapsed Gibbs sampling method in LDA. However, I'm quit confused about the whole method when reviewing the ...
- 1
1
vote
0
answers
35
views
Identification of Latent Variables in Latent Dirichlet Allocation
I am building an Latent Dirichlet Allocation (LDA) model which estimates a version of the Author-Topic model developed by Rosen-Zvi, Griffiths, Steyvers and Smyth. The original paper is here.
The ...
- 11
0
votes
0
answers
44
views
Summarize LDA output using PCA
According to the perplexity' score, optimal number of topics for my dataset is 100. Once I run my LDA algorithm, I'd like to input those topics into an OLS but 100 independent variables is too much. ...
- 1
1
vote
0
answers
67
views
Input for LSI, LDA and HDP?
It seems that the Latent Semantic Indexing (LSI or Latent Semantic Analysis, LSA) can take as input information frequency data (such as TfIdf-weighted space), e.g., according to this documentation ...
- 31
0
votes
0
answers
49
views
How much randomness is required for Gibbs sampling?
I am attempting to parallelize a program that executes hundreds of calls to Mallet's getSampledDistribution method, which is essentially an execution of Gibbs sampling over a topic distribution which ...
0
votes
0
answers
101
views
How to evaluate the performance of recommender systems without having labeled data
I have a huge citation graph of research papers and datasets. So, there is an edge among two items when one of them cites another. So far I've used Node2Vec for creating a dataset recommender system ...
- 1
1
vote
0
answers
65
views
Why does latent dirichlet allocation (LDA) fail when dealing with large and heavy-tailed vocabularies?
I'm reading the 2019 paper Topic Modeling in Embedding Spaces which claims that the embedded topic model improves on these limitations of LDA. But why does LDA have these limitations—why does it fail ...
1
vote
0
answers
101
views
How to use confusion matrix to evaluate accuracy of a topic model (LDA)?
Here's my understanding of topic model, suppose there are 4 topics generated from running LDA from the cleaned corpus(user reviews).
<Topic j: word1(prob.), word2(prob.), word3(prob.)>
Generated ...
- 11
3
votes
0
answers
67
views
Fisherfaces/Linear Discriminat Analysis - What are those faces supposed to be?
I see a lot of weird blue/gray/green faces when I search for "firsherfaces" at Google.
I see faces like this and my question is simple: What are those faces supposed to be?
Are they some ...
- 425
0
votes
0
answers
312
views
PCA whitening and centering in inference/test samples
[cross-posted from SO]
I'm working on speaker identification. I need to take the speaker embeddings from a neural network and apply a few transformations to finally generate the score for verification....
0
votes
0
answers
73
views
Statistical significance of distance between difference of vectors (LDA)
I have an output from Latent Dirichlet Allocation (LDA) which sums up the corpus of about 2,000 documents into 50 topics. The 2,000 documents belong to a category each (in total there are 4 categories,...
2
votes
0
answers
904
views
What are the current approaches to topic modelling (i.e. better than LSA, LDA, LSI)?
I am looking into methods for topic modeling with the purpose of keyword generation. Given a corpus consisting of multiple documents, I would like to get a list of semantically relevant and ...
1
vote
0
answers
62
views
Linear discriminant analysis for non-normal data
I have a classification problem in which the independent variables are not normally distributed.
Is it appropriate to apply linear discriminant analysis for classification? Since lda assumes data to ...
1
vote
0
answers
26
views
Topic modeling with time data
I want to predict the topic of a project tracker narrative entry to identify stages of the project. This entry is written by employees describing what they did in the project.
The projects are ...
- 31
2
votes
1
answer
417
views
Latent Dirichlet Allocation and topic distributions
When reading about the LDA, the generative procedure for a document is often presented as follows:
For each topic $k\in \{1,\ldots, K\}$
Draw a distribution over words $\phi_k\sim \text{Dirichlet}(\...
- 235
1
vote
1
answer
56
views
Understanding original LDA article
I decided to write a different question as a follow up to a comment here about LDA :
Upgrading weight parameters to random variable in Gaussian mixtures
I am trying to read about latent dirichlet ...
- 952
1
vote
1
answer
51
views
In Latent Dirichlet allocation, is the following formula the probability of observing a single document, or an entire corpus?
This is the formula in question:
Source: https://en.wikipedia.org/wiki/Latent_Dirichlet_allocation
- 13
1
vote
1
answer
116
views
Latent Dirichlet Allocation - dimensionality of the Dirichlet prior parameter
I seeking some clarity on the dimensionality of the (hyper)parameter $\eta$ of the "smoothed LDA" model in Section 5.4 of the original paper by Blei, Ng, Jordan (2003), which can be found ...
- 2,630
0
votes
0
answers
857
views
Using KL divergence to select topic from LDA
I have used sklearn's sklearn.decomposition.LatentDirichletAllocation module to model 10 topics in a set of documents.
I have also used the same on a reference text (1 document) and obtained a 1 topic ...
- 142
4
votes
1
answer
147
views
Why choose a Dirichlet distribution in Latent Dirichlet Allocation (LDA)?
I am studying Latent Dirichlet Allocation (LDA), but I don't have much knowledge of stochastic processes.
From Wikipedia:
LDA assumes the following generative process for a corpus $D$
consisting of $...
- 369
0
votes
1
answer
27
views
Suggestions for identifying the most "important" image labels
I have a table with images and their assigned object (i.e. specific sub-parts) and image (i.e. image as a whole) labels. Each image may have multiple object labels but only 2 whole-image labels. I ...
- 11
1
vote
0
answers
22
views
How to know if LDA(Latent Dirichlet Allocation) is applicable to a particular problem? [closed]
Suppose my database has many tables and each table has many columns. I want to use column names of all tables to figure out "major data contents" in my database. Example columns are ...
- 1,027
2
votes
0
answers
118
views
Gibbs sampler of a generative model
I understand what a Gibbs sampler is and I understand how LDA does classification. But I'm unsure how I can generate a Gibbs sampler for an LDA model and how to meld the two concepts. Let's say I ...
- 1,932
5
votes
1
answer
955
views
How to use LDA to classify documents into pre defined topics
LDA is unsupervised and it classifies documents into topics. But, is there a way to make the LDA classify the documents into the predefined (or specific desired) topics.
Below link says we need custom ...
- 847
1
vote
1
answer
559
views
Why VAE need reparameterization trick while LDA does not (both using variational inference for optimization)
Both VAE and LDA (latent Dirichlet allocation) is based on variational inference, and they both try to optimize ELBO objective function
Variational autoencoders use reparameterization so that "...
- 13
2
votes
1
answer
144
views
Updating an unsupervised model but retaining similarity
In my example I am using topic modelling (specifically a version of LDA) although I think avenues for exploring this could relate to other unsupervised techniques like clustering.
I train a model and ...
- 141
0
votes
0
answers
74
views
Topic modelling when the "documents" are not static and new tokens are generated regularly
This question is related to this one and this one. As background, I'm working with Magic: the Gathering decklist data; I'm trying to automatically classify decks into different archetypes based on the ...
- 85
1
vote
0
answers
177
views
parameter estimation on the LDA model
I have a problem with estimating the parameters of $\theta$, and $\phi$ in the Latent Dirichlet Allocation (LDA) model. The article Finding scientific topics has done the estimation of the parameters ...
- 183
1
vote
1
answer
1k
views
Can Latent Dirichlet Allocation (LDA) be used to generate word embeddings?
In the original Word2Vec paper (Efficient Estimation of Word Representations in Vector Space, Mikolov et al. 2013), I came across this phrase:
Many different types of models were proposed for ...
- 113
2
votes
1
answer
749
views
What does each graph quadrant mean with LDAvis plot in R?
I write about the graph explained in this video that shows a big probability mass. It's a web-based interactive visualization of topics estimated using
Latent Dirichlet Allocation that is built using ...
- 135
1
vote
0
answers
48
views
What does each graph cuadrant means with LDA vis plot from R? [duplicate]
I am speeking about this graph, that shows a big probability mass. This is the way that it finally shows with some data I found online and used to understand Latent Dirichlet Allocation topic ...
- 135
1
vote
1
answer
512
views
Inference on Dirichlet hyper-parameter
I'm working on a Gibbs sampler for a (somewhat custom version of) Latent Dirichlet Allocation model.
In short, I have data that comes from a $K$-dimensional Dirichlet-Multinomial distribution, i.e.
$$...
- 153
0
votes
1
answer
164
views
LDA alpha equivalent in structural topic model
I'm using an implementation of the structural topic model (stm), written in R using the stm package.
I want to reduce the number of topics that are prevalent in ...
- 45
2
votes
1
answer
264
views
How to intuitively interpret of topic distributions/coefficients in topic modeling
I know topic modeling such as LDA can provide the distribution of topics in documents, and the distribution of topics represent the important of that topic. However, I'm having a hard time intuitively ...
- 1,030
4
votes
1
answer
665
views
A Bernoulli mixture model with a Dirichlet prior on the parameters
Let's $x_1,...,x_N$ be a set of observation coming from the following generative process:
$$
\boldsymbol{\theta} \sim \text{Dirichlet}(\boldsymbol{\alpha})\qquad\boldsymbol{\theta},\boldsymbol\alpha\...
- 3,056
3
votes
1
answer
465
views
Topic Modeling: How many documents "belong" to a topic?
A frequent question that arises when I present results of topic modeling to novices is: "How many documents belong to topic x?".
As topic modeling is probabilistic, I hesitate giving absolute ...
- 53
0
votes
1
answer
389
views
Inference of the collapsed Gibbs sampling for LDA
I am trying to understand the inference procedure of collapsed Gibbs sampling for LDA model. I refer to this document and LDA wiki page. I cannot figure out how does it simplify the sample equation ...
- 3
0
votes
0
answers
361
views
Custom topic priors in LDA
I've been working with LDA (Latent Dirichlet Allocation topic model) for a while now and I believe I have an intermediate understanding of it.
The unsupervised nature of LDA is one of its big ...
- 153
2
votes
2
answers
4k
views
gensim LdaModel - How to reduce the number of words in each topic?
I'm trying to get more sparse topics (Less overlaps between different topics).
https://radimrehurek.com/gensim/models/ldamodel.html
I know it should be determined by the alpha parameter.
I've ...
2
votes
0
answers
240
views
how to perform statistical analysis on topics produced by topic models?
I have a large corpus of over 1.2 million text documents. After applying LDA to a small sample of the corpus, I have determined the number of topics be 34 using topic coherence score. Here, some ...
user243876
0
votes
2
answers
509
views
using latent dirichlet allocation to reduce the number of dimensions in bag of words model?
Does anyone have experience reducing the dimensions in a traditional bag of words model?
For example, if you want to train a decision tree on a large set of reviews, the size of the vocabulary ...
- 261
1
vote
1
answer
103
views
Prerequisites for Latent Dirichlet Allocation
I have read several "intuitive" introductions to LDA. However, I now want to learn it properly.
I have already read through most of Duda, and that was my introduction to data science. However, I ...
- 102
0
votes
1
answer
743
views
LDA detect new emerging topics
Thanks for stopping by. I have a directional question - I've built a Latent Dirichlet Allocation using Gensims Mallet wrapper. I trained the model once on OldDataSet.csv and measured coherence. I have ...
- 101