3
$\begingroup$

In the stick-breaking construction of Dirichlet (let me base things on Sethuraman's construction - slide 6 of this) do we sample one $\phi$ vector from the base distribution $H$ and use it for sampling $\phi_k$ at each step $k$? This is what I "think" they actually do.

But I am puzzled, as in their paper they mention $H$ is a symmetric Dirichlet distribution over the vocabulary (see the third paragraph of section 2 of this). Which means the dimensionality of $\phi$ would be the size of the vocabulary. Then it is not clear how the index $k$ of $\phi_k$s are mapped to the index of $\beta_k$s. Basically sampling $\beta_k$ will stop if no more stick is left, which means the $k$ index of $\beta_k$ can potentially be much smaller than the size of the $\phi$ vector of size $|\text{Vocabulary}|$ sampled from $H$.

My guess is that they first sample $\phi_k$s, but all they keep from the sample is its corresponding index in $\phi$. Then the sampled $\beta_k$ is just a weight associated with that index. This ways it is clear what each $\beta_k$ corresponds to.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

From what I understand here, they are explaining how to build a Dirichlet process from a base distribution H.

It means that each $\phi_k$ is sampled from H directly. Each $\phi_k$ corresponds to a topic. It is each of these $\phi_k$ which are the size of the vocabulary.

Example : if your vocabulary is of size $N$, each $\phi_k$ can be represented as a vector from $[0, 1]^N$ which corresponds to the probability of each word of appearing in a document of this topic.

The weights are determined through a $GEM(\gamma)$ : even though I'm not sure how I clearly understood how it works, it seems to sequentially assign a weight on each distribution $\phi_k$ until the sum of the weigths is 1, when it assigns then a 0 weight for all subsequent $\phi_k$s

Basically, your $k$ s correspond to the index of the topics, and the $\beta_k$s are drawn indepentendly from the $\phi_k$s.

Don't hesitate to tell me if something is not clear, most of what I tried to explain comes from a paper I worked on last semester on a related subject (Indian Buffet Process)

Improve this answer
$\endgroup$
2
  • $\begingroup$ It makes sense. I do have a question regarding $\delta_{\phi_k}$. They say this is a probability measure concentrated on $\phi_k$. Informally speaking, does it mean it is equal to $\phi_k$? $\endgroup$
    – user3639557
    Commented Apr 11, 2018 at 16:05
  • $\begingroup$ Simply put, $ \delta_{\phi_k} $ is the dirichlet measure $\phi_k$, which means that $G_0$ will be a probability measure on the topic space (as a sum of probability measure that sum to 1 by construction of $\beta_k$). Basically $G_0$ will be the probability measure of chosing a subject among the $k$ subjects $\endgroup$
    – LoicM
    Commented Apr 11, 2018 at 20:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.