Skip to main content

Questions tagged [latent-dirichlet-alloc]

Latent Dirichlet Allocation (LDA) is an unsupervised, statistical approach to document modeling that discovers latent semantic topics in large collections of text. (Do NOT use this for Linear Discriminant Analysis.)

Filter by
Sorted by
Tagged with
1 vote
1 answer
21 views

How does Gibbs sampling end up "sampling from the posterior"? [duplicate]

Let me preface this by saying I have very little experience in statistics. I did my stats courses in undergrad and most of those pertained to things like regression, not Bayesian inference. However, ...
Yannick's user avatar
  • 11
0 votes
1 answer
39 views

About the use of Bayes' rule for continuous valued random variables

I am currently studying the book "An introduction to statistical learning with application in Python" and I am currently at the part 4 of chapter 4 where they explain the general framework ...
Vincent's user avatar
1 vote
0 answers
14 views

Should I evaluate topic coherence of topic model on test set?

I'm a newbie in topic models. From my understanding, topic models like latent dirichlet allocation (LDA) (without empirical Bayes on hyperparameters) performs variational inference to approximate the ...
Kaiwen's user avatar
  • 81
0 votes
0 answers
42 views

Linear Discriminant Analysis (LDA) with continuous value?

I wonder if LDA can be supervised with continuous value. If not, what are the closest variants for such applications?
Johnny Tam's user avatar
0 votes
0 answers
27 views

Reformulating latent dirichlet allocation

can you guys help me solve this equation and go from : to this one : I have tried something but my result gives me this : p(θ, φ, z| α, β) = p(θ, φ, z, w|α, β)/p(w|α, β) the equations are taken from ...
Meyssan's user avatar
0 votes
0 answers
36 views

LDA: sample mean of two distributions are proportional

I came up with a discussion around what does it mean that the sample mean of two distributions are proportional. This discussion started when talking about linear discriminant analysis and how this ...
GGChe's user avatar
  • 165
1 vote
0 answers
112 views

LDA Document Topic Distribution Prediction for Unseen Document

Popular python libraries for topic modeling like gensim or sklearn allow us to predict the topic-distribution for an unseen document, but I have a few questions on what's going on under the hood. I've ...
datasciencewannabe's user avatar
0 votes
0 answers
61 views

Understanding Variational inference for LDA

I am trying to derive from scratch variational inference for LDA. I am following this course: https://home.cs.colorado.edu/~jbg/teaching/CSCI_5622/19a.pdf When computing $p(Z|\Theta)$ they do the ...
sam's user avatar
  • 449
0 votes
1 answer
75 views

How does Gibbs sampling work in Latent Dirichlet Allocation? It seems that only one sample is sampled from the distribution of corpus topics

I'm very new to both Gibbs sampling and LDA. Currently, I'm trying to understand the collapsed Gibbs sampling method in LDA. However, I'm quit confused about the whole method when reviewing the ...
Johnray's user avatar
1 vote
0 answers
31 views

Identification of Latent Variables in Latent Dirichlet Allocation

I am building an Latent Dirichlet Allocation (LDA) model which estimates a version of the Author-Topic model developed by Rosen-Zvi, Griffiths, Steyvers and Smyth. The original paper is here. The ...
Joe Emmens's user avatar
0 votes
0 answers
43 views

Summarize LDA output using PCA

According to the perplexity' score, optimal number of topics for my dataset is 100. Once I run my LDA algorithm, I'd like to input those topics into an OLS but 100 independent variables is too much. ...
Thoms's user avatar
  • 1
1 vote
0 answers
67 views

Input for LSI, LDA and HDP?

It seems that the Latent Semantic Indexing (LSI or Latent Semantic Analysis, LSA) can take as input information frequency data (such as TfIdf-weighted space), e.g., according to this documentation ...
gnm's user avatar
  • 31
0 votes
0 answers
47 views

How much randomness is required for Gibbs sampling?

I am attempting to parallelize a program that executes hundreds of calls to Mallet's getSampledDistribution method, which is essentially an execution of Gibbs sampling over a topic distribution which ...
Fledgling Pidgeon's user avatar
0 votes
0 answers
95 views

How to evaluate the performance of recommender systems without having labeled data

I have a huge citation graph of research papers and datasets. So, there is an edge among two items when one of them cites another. So far I've used Node2Vec for creating a dataset recommender system ...
Morteza's user avatar
1 vote
0 answers
64 views

Why does latent dirichlet allocation (LDA) fail when dealing with large and heavy-tailed vocabularies?

I'm reading the 2019 paper Topic Modeling in Embedding Spaces which claims that the embedded topic model improves on these limitations of LDA. But why does LDA have these limitations—why does it fail ...
seanmachinelearning's user avatar
1 vote
0 answers
99 views

How to use confusion matrix to evaluate accuracy of a topic model (LDA)?

Here's my understanding of topic model, suppose there are 4 topics generated from running LDA from the cleaned corpus(user reviews). <Topic j: word1(prob.), word2(prob.), word3(prob.)> Generated ...
Locke's user avatar
  • 11
3 votes
0 answers
64 views

Fisherfaces/Linear Discriminat Analysis - What are those faces supposed to be?

I see a lot of weird blue/gray/green faces when I search for "firsherfaces" at Google. I see faces like this and my question is simple: What are those faces supposed to be? Are they some ...
euraad's user avatar
  • 415
0 votes
0 answers
306 views

PCA whitening and centering in inference/test samples

[cross-posted from SO] I'm working on speaker identification. I need to take the speaker embeddings from a neural network and apply a few transformations to finally generate the score for verification....
Zabir Al Nazi's user avatar
0 votes
0 answers
73 views

Statistical significance of distance between difference of vectors (LDA)

I have an output from Latent Dirichlet Allocation (LDA) which sums up the corpus of about 2,000 documents into 50 topics. The 2,000 documents belong to a category each (in total there are 4 categories,...
blackmamba's user avatar
2 votes
0 answers
888 views

What are the current approaches to topic modelling (i.e. better than LSA, LDA, LSI)?

I am looking into methods for topic modeling with the purpose of keyword generation. Given a corpus consisting of multiple documents, I would like to get a list of semantically relevant and ...
Martin Wunderlich's user avatar
1 vote
0 answers
61 views

Linear discriminant analysis for non-normal data

I have a classification problem in which the independent variables are not normally distributed. Is it appropriate to apply linear discriminant analysis for classification? Since lda assumes data to ...
Ankit Dwivedi's user avatar
1 vote
0 answers
25 views

Topic modeling with time data

I want to predict the topic of a project tracker narrative entry to identify stages of the project. This entry is written by employees describing what they did in the project. The projects are ...
Jess's user avatar
  • 31
2 votes
1 answer
399 views

Latent Dirichlet Allocation and topic distributions

When reading about the LDA, the generative procedure for a document is often presented as follows: For each topic $k\in \{1,\ldots, K\}$ Draw a distribution over words $\phi_k\sim \text{Dirichlet}(\...
Tyler D's user avatar
  • 225
1 vote
1 answer
55 views

Understanding original LDA article

I decided to write a different question as a follow up to a comment here about LDA : Upgrading weight parameters to random variable in Gaussian mixtures I am trying to read about latent dirichlet ...
Thomas's user avatar
  • 900
1 vote
1 answer
49 views

In Latent Dirichlet allocation, is the following formula the probability of observing a single document, or an entire corpus?

This is the formula in question: Source: https://en.wikipedia.org/wiki/Latent_Dirichlet_allocation
Bob Odenkirk's user avatar
1 vote
1 answer
114 views

Latent Dirichlet Allocation - dimensionality of the Dirichlet prior parameter

I seeking some clarity on the dimensionality of the (hyper)parameter $\eta$ of the "smoothed LDA" model in Section 5.4 of the original paper by Blei, Ng, Jordan (2003), which can be found ...
microhaus's user avatar
  • 2,560
0 votes
0 answers
836 views

Using KL divergence to select topic from LDA

I have used sklearn's sklearn.decomposition.LatentDirichletAllocation module to model 10 topics in a set of documents. I have also used the same on a reference text (1 document) and obtained a 1 topic ...
Migos's user avatar
  • 142
4 votes
1 answer
135 views

Why choose a Dirichlet distribution in Latent Dirichlet Allocation (LDA)?

I am studying Latent Dirichlet Allocation (LDA), but I don't have much knowledge of stochastic processes. From Wikipedia: LDA assumes the following generative process for a corpus $D$ consisting of $...
Mark's user avatar
  • 307
0 votes
1 answer
27 views

Suggestions for identifying the most "important" image labels

I have a table with images and their assigned object (i.e. specific sub-parts) and image (i.e. image as a whole) labels. Each image may have multiple object labels but only 2 whole-image labels. I ...
nlapidot's user avatar
1 vote
0 answers
22 views

How to know if LDA(Latent Dirichlet Allocation) is applicable to a particular problem? [closed]

Suppose my database has many tables and each table has many columns. I want to use column names of all tables to figure out "major data contents" in my database. Example columns are ...
etang's user avatar
  • 997
2 votes
0 answers
112 views

Gibbs sampler of a generative model

I understand what a Gibbs sampler is and I understand how LDA does classification. But I'm unsure how I can generate a Gibbs sampler for an LDA model and how to meld the two concepts. Let's say I ...
Christian's user avatar
  • 1,912
5 votes
1 answer
925 views

How to use LDA to classify documents into pre defined topics

LDA is unsupervised and it classifies documents into topics. But, is there a way to make the LDA classify the documents into the predefined (or specific desired) topics. Below link says we need custom ...
tjt's user avatar
  • 827
1 vote
1 answer
545 views

Why VAE need reparameterization trick while LDA does not (both using variational inference for optimization)

Both VAE and LDA (latent Dirichlet allocation) is based on variational inference, and they both try to optimize ELBO objective function Variational autoencoders use reparameterization so that "...
user3462510's user avatar
2 votes
1 answer
129 views

Updating an unsupervised model but retaining similarity

In my example I am using topic modelling (specifically a version of LDA) although I think avenues for exploring this could relate to other unsupervised techniques like clustering. I train a model and ...
Jaccar's user avatar
  • 141
0 votes
0 answers
74 views

Topic modelling when the "documents" are not static and new tokens are generated regularly

This question is related to this one and this one. As background, I'm working with Magic: the Gathering decklist data; I'm trying to automatically classify decks into different archetypes based on the ...
John Doe's user avatar
1 vote
0 answers
175 views

parameter estimation on the LDA model

I have a problem with estimating the parameters of $\theta$, and $\phi$ in the Latent Dirichlet Allocation (LDA) model. The article Finding scientific topics has done the estimation of the parameters ...
Siamak Abdi's user avatar
1 vote
1 answer
1k views

Can Latent Dirichlet Allocation (LDA) be used to generate word embeddings?

In the original Word2Vec paper (Efficient Estimation of Word Representations in Vector Space, Mikolov et al. 2013), I came across this phrase: Many different types of models were proposed for ...
NeuronQ's user avatar
  • 113
2 votes
1 answer
720 views

What does each graph quadrant mean with LDAvis plot in R?

I write about the graph explained in this video that shows a big probability mass. It's a web-based interactive visualization of topics estimated using Latent Dirichlet Allocation that is built using ...
MelaniaCB's user avatar
  • 135
1 vote
0 answers
48 views

What does each graph cuadrant means with LDA vis plot from R? [duplicate]

I am speeking about this graph, that shows a big probability mass. This is the way that it finally shows with some data I found online and used to understand Latent Dirichlet Allocation topic ...
MelaniaCB's user avatar
  • 135
1 vote
1 answer
503 views

Inference on Dirichlet hyper-parameter

I'm working on a Gibbs sampler for a (somewhat custom version of) Latent Dirichlet Allocation model. In short, I have data that comes from a $K$-dimensional Dirichlet-Multinomial distribution, i.e. $$...
yassem's user avatar
  • 153
0 votes
1 answer
158 views

LDA alpha equivalent in structural topic model

I'm using an implementation of the structural topic model (stm), written in R using the stm package. I want to reduce the number of topics that are prevalent in ...
James's user avatar
  • 25
2 votes
1 answer
259 views

How to intuitively interpret of topic distributions/coefficients in topic modeling

I know topic modeling such as LDA can provide the distribution of topics in documents, and the distribution of topics represent the important of that topic. However, I'm having a hard time intuitively ...
Jane Sully's user avatar
  • 1,030
4 votes
1 answer
655 views

A Bernoulli mixture model with a Dirichlet prior on the parameters

Let's $x_1,...,x_N$ be a set of observation coming from the following generative process: $$ \boldsymbol{\theta} \sim \text{Dirichlet}(\boldsymbol{\alpha})\qquad\boldsymbol{\theta},\boldsymbol\alpha\...
alberto's user avatar
  • 3,056
3 votes
1 answer
447 views

Topic Modeling: How many documents "belong" to a topic?

A frequent question that arises when I present results of topic modeling to novices is: "How many documents belong to topic x?". As topic modeling is probabilistic, I hesitate giving absolute ...
abitter's user avatar
  • 53
0 votes
1 answer
362 views

Inference of the collapsed Gibbs sampling for LDA

I am trying to understand the inference procedure of collapsed Gibbs sampling for LDA model. I refer to this document and LDA wiki page. I cannot figure out how does it simplify the sample equation ...
Sin.Sun's user avatar
0 votes
0 answers
348 views

Custom topic priors in LDA

I've been working with LDA (Latent Dirichlet Allocation topic model) for a while now and I believe I have an intermediate understanding of it. The unsupervised nature of LDA is one of its big ...
yassem's user avatar
  • 153
2 votes
2 answers
4k views

gensim LdaModel - How to reduce the number of words in each topic?

I'm trying to get more sparse topics (Less overlaps between different topics). https://radimrehurek.com/gensim/models/ldamodel.html I know it should be determined by the alpha parameter. I've ...
JohnSnowTheDeveloper's user avatar
2 votes
0 answers
236 views

how to perform statistical analysis on topics produced by topic models?

I have a large corpus of over 1.2 million text documents. After applying LDA to a small sample of the corpus, I have determined the number of topics be 34 using topic coherence score. Here, some ...
user avatar
0 votes
2 answers
501 views

using latent dirichlet allocation to reduce the number of dimensions in bag of words model?

Does anyone have experience reducing the dimensions in a traditional bag of words model? For example, if you want to train a decision tree on a large set of reviews, the size of the vocabulary ...
barker's user avatar
  • 261
1 vote
1 answer
103 views

Prerequisites for Latent Dirichlet Allocation

I have read several "intuitive" introductions to LDA. However, I now want to learn it properly. I have already read through most of Duda, and that was my introduction to data science. However, I ...
Avatrin's user avatar
  • 92