Return to Question

deleted 2 characters in body; edited tags

Source Link

edited Feb 28, 2017 at 18:39

47.6k
33
179
293

I am trying to figure out the details of LDA and have been stuck for a while now. While reading the paper by Blei, I came across this -

Latent Dirichlet allocation (LDA) is a generative probabilistic model of a corpus. The basic idea is that documents are represented as random mixtures over latent topics, where each topic is characterized by a distribution over words. LDA assumes the following generative process for each document w in a corpus D:

Choose N ∼ Poisson(ξ)

Choose θ ∼ Dir(α)

For each of the N words w_n:

Choose a topic z_n ∼ Multinomial(θ).

Choose a word w_n from p(w_n |z_n,β), a multinomial probability conditioned on the topic z_n.

When they mention Choose θ ∼ Dir(α), they mean that the probabilities come from a Dirichlet distribution and in case of an observation (or any number of obs), we update the α and the probabilities change accordingly. And when they say Choose N ∼ Poisson(ξ), they mean the number of words in each document can be sampled from a Poisson distribution, that is if I have the number of words in each document they will follow a Poisson distribution. So this part I get (or correct me if I have gotten the details wrong).

Gaps in my understanding:

I want to understand what the authors mean when they say choose a topic z_n ~ Multinomial(θ). From my understanding of the multinomial distribution parameterized by θ, you get the probability of a particular distribution of events (ex: picking a ball from a bag with replacement n times). So how do we choose a topic for each word using a multinomial distribution?
From what I understand each document is randomly assigned topic probabilities and each topic is assigned word probabilities and is updated later on. My question is how are these probabilities updated?

I am trying to figure out the details of LDA and have been stuck for a while now. While reading the paper by Blei, I came across this -

Latent Dirichlet allocation (LDA) is a generative probabilistic model of a corpus. The basic idea is that documents are represented as random mixtures over latent topics, where each topic is characterized by a distribution over words. LDA assumes the following generative process for each document w in a corpus D:

Choose N ∼ Poisson(ξ)

Choose θ ∼ Dir(α)

For each of the N words w_n:

Choose a topic z_n ∼ Multinomial(θ).

Choose a word w_n from p(w_n |z_n,β), a multinomial probability conditioned on the topic z_n.

Gaps in my understanding:

I want to understand what the authors mean when they say choose a topic z_n ~ Multinomial(θ). From my understanding of the multinomial distribution parameterized by θ, you get the probability of a particular distribution of events (ex: picking a ball from a bag with replacement n times). So how do we choose a topic for each word using a multinomial distribution?
From what I understand each document is randomly assigned topic probabilities and each topic is assigned word probabilities and is updated later on. My question is how are these probabilities updated?

I am trying to figure out the details of LDA and have been stuck for a while now. While reading the paper by Blei, I came across this -

Latent Dirichlet allocation (LDA) is a generative probabilistic model of a corpus. The basic idea is that documents are represented as random mixtures over latent topics, where each topic is characterized by a distribution over words. LDA assumes the following generative process for each document w in a corpus D:

Choose N ∼ Poisson(ξ)

Choose θ ∼ Dir(α)

For each of the N words w_n:

Choose a topic z_n ∼ Multinomial(θ).

Choose a word w_n from p(w_n |z_n,β), a multinomial probability conditioned on the topic z_n.

Gaps in my understanding:

I want to understand what the authors mean when they say choose a topic z_n ~ Multinomial(θ). From my understanding of the multinomial distribution parameterized by θ, you get the probability of a particular distribution of events (ex: picking a ball from a bag with replacement n times). So how do we choose a topic for each word using a multinomial distribution?
From what I understand each document is randomly assigned topic probabilities and each topic is assigned word probabilities and is updated later on. My question is how are these probabilities updated?

Source Link

asked Dec 15, 2016 at 15:11

Clock Slave

1.1k
1
10
22

Topic Models: Latent Dirichlet Allocations

I am trying to figure out the details of LDA and have been stuck for a while now. While reading the paper by Blei, I came across this -

Latent Dirichlet allocation (LDA) is a generative probabilistic model of a corpus. The basic idea is that documents are represented as random mixtures over latent topics, where each topic is characterized by a distribution over words. LDA assumes the following generative process for each document w in a corpus D:

Choose N ∼ Poisson(ξ)

Choose θ ∼ Dir(α)

For each of the N words w_n:

Choose a topic z_n ∼ Multinomial(θ).

Choose a word w_n from p(w_n |z_n,β), a multinomial probability conditioned on the topic z_n.

Gaps in my understanding:

I want to understand what the authors mean when they say choose a topic z_n ~ Multinomial(θ). From my understanding of the multinomial distribution parameterized by θ, you get the probability of a particular distribution of events (ex: picking a ball from a bag with replacement n times). So how do we choose a topic for each word using a multinomial distribution?
From what I understand each document is randomly assigned topic probabilities and each topic is assigned word probabilities and is updated later on. My question is how are these probabilities updated?

dirichlet-distribution topic-models lda