Running Latent Dirichlet Allocation (LDA) on word counts I have difficulties understanding the VB implementation lda-c. In particular, the method expects as input a bag-of-words representation of documents, where distinct words appearing in a document are mapped to the number of their occurence in that document. The generative model, however, seems to be specified in terms of a sequence of words per document denoted by $w_{d, n} \in \{1,\ldots,V\}$ for the $n$-th word taken from our vocabulary of size $V$ in the $d$-th document. Each word is assigned a latent topic variable $z_{d,n} \in \{1, \ldots, K\}$. Given $z_{d,n}$ and word distributions $\beta_k$ per topic $k$, the likelihood is written as
$$
P(w_{d,n} \mid z_{d,n}, \beta) = \beta_{w_{d,n}, z_{d,n}}
$$
This formulation seems to specifically rely on knowing each word in the document.
So if I instead observe only a histogram of words $n_d$ where $n_{d,j}$ is the number of times word $j$ appears in document $d$, how would the likelihood $P(n_{d,n} \mid z_d, \beta)$ look like? 
It feels like there should be a way to rewrite $P(w \mid z, \beta)$ as $P(n \mid z, \beta)$ as the word sequence is really exchangeable.
 A: When you only have word counts but no documents you can generate a document from the word counts by putting the words according to their frequency in random order into the document. Then the algorithm runs smoothly without rewriting.
A: In the original LDA paper the authors note that the variational parameters $\gamma$ and $\phi$ are functions of the words $\mathbf{w}$ because the optimization of the lower bound is done for fixed $\mathbf{w}$. This means that for document $d$, topic $i$, and word positions in the document $n_j$ and $n_k$, if $w_{dn_j} = w_{dn_k}$, that is words $n_j$ and $n_k$ are the same, then:
\begin{equation}
\phi_{dn_ji}(w_{dn_j}) = \phi_{dn_ki}(w_{dn_k})
\end{equation}
The M-step update for $\beta$ is 
\begin{equation}
\beta_{ij} \propto \sum_{d=1}^M\sum_{n=1}^{N_d} \phi_{dni}w_{dn}^j
\end{equation}
Then we see that the inner sum is counting the number of times word with identity $j$ appears in document $d$ and multiplying that by some $\phi_{dji}$. So we can rewrite the M-step update as:
\begin{equation}
\beta_{ij} \propto \sum_{d=1}^M \phi_{dji}n_{dj}
\end{equation}
where $n_{dj}$ is the number of times the word $j$ appears in document $d$. Notice that for each document we don't have as many $\phi$ values as there are words in the document. There are only as many distinct $\phi$ values as there are distinct words. 
This is why lda-c expects bag-of-words input.
