1
$\begingroup$

I'm trying to understand the following plate notation which is used a lot as an example of topic model to introduce variational methods, etc.

I wanted to ask if my understanding is correctly depicted in the attempt sketched below in order to parse and unfold the plate notation of the topic model represented?

I have omitted the $\alpha$ and $\theta$ parameters but please feel free to add them in case it makes the example more understandable and easier for others to read as well.

Thanks! enter image description here

New diagram describing how to parse plate notation of LDA topic model taking into account @Tracy Chen comments

enter image description here

Improve this question
$\endgroup$
1
  • $\begingroup$ So there's no one here who has had the same question or had gone through the same notation before? I just need someone to indicate my mistakes, that's all. $\endgroup$
    – Kirk Walla
    Dec 2, 2019 at 12:12

1 Answer 1

Reset to default
1
$\begingroup$
  1. Which topic models are you referring to here? If you mean Latent Dirichlet Allocation, please note that your plate diagram is inaccurate, as in LDA we assume each document has its own $\theta_d$. For the plate diagram of LDA, please see this paper.
  2. Let's assume you are drawing another type of topic model, in your model, the assumption is that the whole corpus shares the same topic proportion $\theta$, while in each document, it only contains one topic $z_d$. Then you are right about the repetitions of $z_d$, however, for each word, you don't need to repeat $\beta_K$ for $K$ times, as it can only come from one topic distribution ($\beta_k$, while $k=z_d$ as $z_d$ is only an index).
Improve this answer
$\endgroup$
7
  • $\begingroup$ thanks for the clarifications. Regarding comment 1.), yes topic model I was referring to is the LDA as shown here. I think the plate diagram from the link provided is correct? Regarding comment 2.), I believe this clarified a few things for me and I've edited the OP with a new hand drawn diagram to reflect your comment. Let me know if that's correct according to your comment? $\endgroup$
    – Kirk Walla
    Mar 2, 2020 at 16:07
  • $\begingroup$ Yes, now your hand-drawn diagram is right. However, you were drawing the wrong diagram in the first place. On the slides you attach here, it actually first shows the plate diagram of mixture models on slide 2, and then it shows the LDA plate diagram on slide 10, you can read the slide 11 for the differences of those two models. $\endgroup$
    – TracyYXChen
    Mar 2, 2020 at 17:06
  • $\begingroup$ You're absolutely right, good catch, in that case the hand drawn diagram should be modified in order each $z_{d}$ to get its own $\theta$ and not a shared one as it is represented now? $\endgroup$
    – Kirk Walla
    Mar 2, 2020 at 17:51
  • $\begingroup$ The notation should be $\theta_d$, $z_{nd}$, $w_{nd}$, i.e. a document has a $\theta$, each word $w$ has its own topic assignment $z$. (Not each $z_{nd}$ gets its own $\theta$) $\endgroup$
    – TracyYXChen
    Mar 2, 2020 at 18:04
  • $\begingroup$ Yes. that's exactly what I meant (if I understand you correctly) instead of having a single vector $\theta_{1,\ldots,K}$ where each element/value $k$ representing all the document $z_{d}$, now each $z_{1\ldots,d}$ gets its own $\theta_{1,\ldots,K}$, where supposedly we have chosen $K=10$ topics, for the sake of the example. $\endgroup$
    – Kirk Walla
    Mar 2, 2020 at 18:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.