Questions tagged [latent-dirichlet-alloc]

Latent Dirichlet Allocation (LDA) is an unsupervised, statistical approach to document modeling that discovers latent semantic topics in large collections of text. (Do NOT use this for Linear Discriminant Analysis.)

Filter by
Sorted by
Tagged with
0 votes
0 answers
7 views

How to interpret unsupervised results

I have applied some unsupervised techniques on text data in order to do topic modeling such as LDA. Once training is completed LDA produces associations between documents and topics. Now my question ...
user avatar
  • 1
0 votes
0 answers
7 views

Why does latent dirichlet allocation (LDA) fail when dealing with large and heavy-tailed vocabularies?

I'm reading the 2019 paper Topic Modeling in Embedding Spaces which claims that the embedded topic model improves on these limitations of LDA. But why does LDA have these limitations—why does it fail ...
user avatar
0 votes
0 answers
43 views

Non-Dirichlet Prior for $Cat(\theta)$ parameter that can tractably be integrated out (for Latent Dirichlet Analysis)?

In LDA Topic Models, it is standard to 'integrate out' the $\theta$ parameter, which contains a document's Categorical probabilities of drawing each topic. QUESTION If one uses the standard Dirichlet ...
user avatar
0 votes
0 answers
31 views

Calculate a measure based on LDA topics and Hellinger distance

I am trying to calculate some sort of ambiguity/heterogeneity measure from text based topic probabilities from a Latent Dirichlet Allocation model and the Hellinger distance between the topic ...
user avatar
1 vote
0 answers
19 views

How to use confusion matrix to evaluate accuracy of a topic model (LDA)?

Here's my understanding of topic model, suppose there are 4 topics generated from running LDA from the cleaned corpus(user reviews). <Topic j: word1(prob.), word2(prob.), word3(prob.)> Generated ...
user avatar
  • 11
0 votes
0 answers
49 views

Proper way to evaluate different LDA models on different parameters

Radim Řehůřek from Gensim writes on his page: I suggest the following way to choose iterations and passes. First, enable logging (as described in many Gensim tutorials), and set eval_every = 1 in ...
user avatar
  • 1
0 votes
0 answers
63 views

Which Gensim's LDA parameters to test for finding optimal model?

For now I want to test these parameters and their ranges: num_topics (10 to 500; otherwise process of training and Coherence computing gets too long) alpha: ...
user avatar
  • 1
0 votes
0 answers
22 views

What is a key difference(if any) between Latent Class Analysis and Latent Dirichlet analysis

I am trying to use latent class analysis for clustering to improve upon K-means and was curious how it relates to latent dirichlet analysis
user avatar
0 votes
0 answers
17 views

Text analysis how to handle very similar n-grams

I have stemmed and tf-idf'd some text for use with LDA topic extraction. My task is to examine some of the top n-gram > 2 results. Initially, I had presumed that ...
user avatar
0 votes
0 answers
27 views

Reference Request: Variational Expectation-Maximization algorithm for Latent Dirichlet Allocation with an added time component

This link has a pretty good runthrough on the variational inference (via variational E-M) for LDA with calculations expanded and explained. I am now considering a modified LDA which adds a time ...
user avatar
  • 1
3 votes
0 answers
30 views

Fisherfaces/Linear Discriminat Analysis - What are those faces supposed to be?

I see a lot of weird blue/gray/green faces when I search for "firsherfaces" at Google. I see faces like this and my question is simple: What are those faces supposed to be? Are they some ...
user avatar
  • 173
0 votes
0 answers
33 views

PCA whitening and centering in inference/test samples

[cross-posted from SO] I'm working on speaker identification. I need to take the speaker embeddings from a neural network and apply a few transformations to finally generate the score for verification....
user avatar
0 votes
0 answers
37 views

Statistical significance of distance between difference of vectors (LDA)

I have an output from Latent Dirichlet Allocation (LDA) which sums up the corpus of about 2,000 documents into 50 topics. The 2,000 documents belong to a category each (in total there are 4 categories,...
user avatar
2 votes
0 answers
650 views

What are the current approaches to topic modelling (i.e. better than LSA, LDA, LSI)?

I am looking into methods for topic modeling with the purpose of keyword generation. Given a corpus consisting of multiple documents, I would like to get a list of semantically relevant and ...
user avatar
  • 173
1 vote
0 answers
24 views

Linear discriminant analysis for non-normal data

I have a classification problem in which the independent variables are not normally distributed. Is it appropriate to apply linear discriminant analysis for classification? Since lda assumes data to ...
user avatar
1 vote
0 answers
19 views

Topic modeling with time data

I want to predict the topic of a project tracker narrative entry to identify stages of the project. This entry is written by employees describing what they did in the project. The projects are ...
user avatar
  • 31
1 vote
1 answer
181 views

Latent Dirichlet Allocation and topic distributions

When reading about the LDA, the generative procedure for a document is often presented as follows: For each topic $k\in \{1,\ldots, K\}$ Draw a distribution over words $\phi_k\sim \text{Dirichlet}(\...
user avatar
  • 145
1 vote
1 answer
48 views

Understanding original LDA article

I decided to write a different question as a follow up to a comment here about LDA : Upgrading weight parameters to random variable in Gaussian mixtures I am trying to read about latent dirichlet ...
user avatar
  • 663
1 vote
1 answer
29 views

In Latent Dirichlet allocation, is the following formula the probability of observing a single document, or an entire corpus?

This is the formula in question: Source: https://en.wikipedia.org/wiki/Latent_Dirichlet_allocation
user avatar
1 vote
1 answer
74 views

Latent Dirichlet Allocation - dimensionality of the Dirichlet prior parameter

I seeking some clarity on the dimensionality of the (hyper)parameter $\eta$ of the "smoothed LDA" model in Section 5.4 of the original paper by Blei, Ng, Jordan (2003), which can be found ...
user avatar
  • 2,206
0 votes
0 answers
447 views

Using KL divergence to select topic from LDA

I have used sklearn's sklearn.decomposition.LatentDirichletAllocation module to model 10 topics in a set of documents. I have also used the same on a reference text (1 document) and obtained a 1 topic ...
user avatar
  • 142
1 vote
0 answers
29 views

Why choose a Dirichlet distribution in Latent Dirichlet Allocation (LDA)?

I am studying Latent Dirichlet Allocation (LDA), but I don't have much knowledge of stochastic processes. From Wikipedia: LDA assumes the following generative process for a corpus $D$ consisting of $...
user avatar
  • 187
0 votes
1 answer
26 views

Suggestions for identifying the most "important" image labels

I have a table with images and their assigned object (i.e. specific sub-parts) and image (i.e. image as a whole) labels. Each image may have multiple object labels but only 2 whole-image labels. I ...
user avatar
1 vote
0 answers
18 views

How to know if LDA(Latent Dirichlet Allocation) is applicable to a particular problem? [closed]

Suppose my database has many tables and each table has many columns. I want to use column names of all tables to figure out "major data contents" in my database. Example columns are ...
user avatar
  • 815
2 votes
0 answers
61 views

Gibbs sampler of a generative model

I understand what a Gibbs sampler is and I understand how LDA does classification. But I'm unsure how I can generate a Gibbs sampler for an LDA model and how to meld the two concepts. Let's say I ...
user avatar
  • 1,472
5 votes
1 answer
442 views

How to use LDA to classify documents into pre defined topics

LDA is unsupervised and it classifies documents into topics. But, is there a way to make the LDA classify the documents into the predefined (or specific desired) topics. Below link says we need custom ...
user avatar
  • 717
0 votes
1 answer
329 views

Why VAE need reparameterization trick while LDA does not (both using variational inference for optimization)

Both VAE and LDA (latent Dirichlet allocation) is based on variational inference, and they both try to optimize ELBO objective function Variational autoencoders use reparameterization so that "...
user avatar
2 votes
1 answer
59 views

Updating an unsupervised model but retaining similarity

In my example I am using topic modelling (specifically a version of LDA) although I think avenues for exploring this could relate to other unsupervised techniques like clustering. I train a model and ...
user avatar
  • 141
0 votes
0 answers
45 views

Topic modelling when the "documents" are not static and new tokens are generated regularly

This question is related to this one and this one. As background, I'm working with Magic: the Gathering decklist data; I'm trying to automatically classify decks into different archetypes based on the ...
user avatar
1 vote
0 answers
112 views

parameter estimation on the LDA model

I have a problem with estimating the parameters of $\theta$, and $\phi$ in the Latent Dirichlet Allocation (LDA) model. The article Finding scientific topics has done the estimation of the parameters ...
user avatar
1 vote
1 answer
595 views

Can Latent Dirichlet Allocation (LDA) be used to generate word embeddings?

In the original Word2Vec paper (Efficient Estimation of Word Representations in Vector Space, Mikolov et al. 2013), I came across this phrase: Many different types of models were proposed for ...
user avatar
  • 113
2 votes
1 answer
410 views

What does each graph quadrant mean with LDAvis plot in R?

I write about the graph explained in this video that shows a big probability mass. It's a web-based interactive visualization of topics estimated using Latent Dirichlet Allocation that is built using ...
user avatar
1 vote
0 answers
48 views

What does each graph cuadrant means with LDA vis plot from R? [duplicate]

I am speeking about this graph, that shows a big probability mass. This is the way that it finally shows with some data I found online and used to understand Latent Dirichlet Allocation topic ...
user avatar
1 vote
1 answer
256 views

Inference on Dirichlet hyper-parameter

I'm working on a Gibbs sampler for a (somewhat custom version of) Latent Dirichlet Allocation model. In short, I have data that comes from a $K$-dimensional Dirichlet-Multinomial distribution, i.e. $$...
user avatar
  • 133
0 votes
1 answer
100 views

LDA alpha equivalent in structural topic model

I'm using an implementation of the structural topic model (stm), written in R using the stm package. I want to reduce the number of topics that are prevalent in ...
user avatar
  • 25
1 vote
1 answer
148 views

How to intuitively interpret of topic distributions/coefficients in topic modeling

I know topic modeling such as LDA can provide the distribution of topics in documents, and the distribution of topics represent the important of that topic. However, I'm having a hard time intuitively ...
user avatar
3 votes
1 answer
432 views

A Bernoulli mixture model with a Dirichlet prior on the parameters

Let's $x_1,...,x_N$ be a set of observation coming from the following generative process: $$ \boldsymbol{\theta} \sim \text{Dirichlet}(\boldsymbol{\alpha})\qquad\boldsymbol{\theta},\boldsymbol\alpha\...
user avatar
  • 2,696
3 votes
1 answer
213 views

Topic Modeling: How many documents "belong" to a topic?

A frequent question that arises when I present results of topic modeling to novices is: "How many documents belong to topic x?". As topic modeling is probabilistic, I hesitate giving absolute ...
user avatar
  • 53
0 votes
1 answer
196 views

Inference of the collapsed Gibbs sampling for LDA

I am trying to understand the inference procedure of collapsed Gibbs sampling for LDA model. I refer to this document and LDA wiki page. I cannot figure out how does it simplify the sample equation ...
user avatar
0 votes
0 answers
215 views

Custom topic priors in LDA

I've been working with LDA (Latent Dirichlet Allocation topic model) for a while now and I believe I have an intermediate understanding of it. The unsupervised nature of LDA is one of its big ...
user avatar
  • 133
2 votes
2 answers
3k views

gensim LdaModel - How to reduce the number of words in each topic?

I'm trying to get more sparse topics (Less overlaps between different topics). https://radimrehurek.com/gensim/models/ldamodel.html I know it should be determined by the alpha parameter. I've ...
user avatar
2 votes
0 answers
108 views

how to perform statistical analysis on topics produced by topic models?

I have a large corpus of over 1.2 million text documents. After applying LDA to a small sample of the corpus, I have determined the number of topics be 34 using topic coherence score. Here, some ...
user avatar
0 votes
1 answer
334 views

using latent dirichlet allocation to reduce the number of dimensions in bag of words model?

Does anyone have experience reducing the dimensions in a traditional bag of words model? For example, if you want to train a decision tree on a large set of reviews, the size of the vocabulary ...
user avatar
  • 219
1 vote
1 answer
70 views

Prerequisites for Latent Dirichlet Allocation

I have read several "intuitive" introductions to LDA. However, I now want to learn it properly. I have already read through most of Duda, and that was my introduction to data science. However, I ...
user avatar
  • 112
0 votes
1 answer
655 views

LDA detect new emerging topics

Thanks for stopping by. I have a directional question - I've built a Latent Dirichlet Allocation using Gensims Mallet wrapper. I trained the model once on OldDataSet.csv and measured coherence. I have ...
user avatar
  • 101
2 votes
3 answers
10k views

What is the formula for c_v coherence?

I've recently been playing around with Gensim LDAModel. I use coherence to evaluate the results. Gensim offers a few coherence measures. This includes c_v and ...
user avatar
3 votes
1 answer
987 views

Choosing the number of topics in topic modeling with multiple "elbows" in the coherence plot

When plotting the number of topics on the x-axis and the coherence score on the y-axis, I had expected to see an "elbow" (for example, here and here). In this case, however, the plot does not have a ...
user avatar
1 vote
0 answers
160 views

How to interpret LDA (Latent Dirichlet Allocation)?

Say I want to run topic modeling with LDA on The 20 newsgroups text dataset. So basically a dataset with texts where every text belongs to one of 20 categories. I want the LDA to split the documents ...
user avatar
1 vote
1 answer
284 views

When DV is dichotomous and IV is ordinal, should I use LPA or LCA?

Suppose my dependent/outcome variable is composed of 6 vignettes that has dichotomous outcomes of "Yes" (1) or "No" (0). Normally, I can just get a sum-score total so that it's a score from 0 (...
user avatar
  • 191
1 vote
0 answers
143 views

Why does perplexity change with different ranges of k?

I ran a 5-fold cross-validation in R to calculate LDA perplexity for k = 2:9 using a 10% sample of my data. The output was: ...
user avatar