I understand what a Gibbs sampler is and I understand how LDA does classification. But I'm unsure how I can generate a Gibbs sampler for an LDA model and how to meld the two concepts. Let's say I define a generative model as follows:

  • For each sentence $m$:
    • Sample sentiment probabilities $\theta_m \sim Dirichlet(\alpha)$
    • For each word $n$:
      • Sample a sentiment $s_{mn} \sim Multinomial(\theta_m)$
      • Sample a word $t_{mn} \sim Multinomial(\beta)$

Where $\alpha, \beta$ are fixed hyperparameters.

I am to write out the conditional probabilities for the Gibbs sampler. But not sure how there are multiple conditional probabilities that construct a Gibbs sampler. Isn't there just a single equation? There might be some major misunderstanding of some concept here.

  • 1
    $\begingroup$ Steyvers et al. (which I also believe were the first to publish using Gibbs for LDA) gives an overview of it here, see equation (3): cocosci.princeton.edu/tom/papers/SteyversGriffiths.pdf. It isn't too hard to implement once you wrap your head around it. It's rather slow though. Collapsed Gibbs Sampling is a better option (or skipping right to the variational inference algos.).You will sample "topics" not sentiment. There are versions that incorporate sentiment though... $\endgroup$
    – EzioBosso
    Oct 17 '20 at 4:31
  • $\begingroup$ I can expand on their equation if it isn't clear. Let me know. $\endgroup$
    – EzioBosso
    Oct 17 '20 at 4:33
  • $\begingroup$ @EzioBosso I updated my question to be a bit more specific. Your link was a great source. I see what the conditional probability is. But my task is to write the set of conditional probabilities. Please see updated question. $\endgroup$
    – Christian
    Oct 18 '20 at 4:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.