I'm working on a Gibbs sampler for a (somewhat custom version of) Latent Dirichlet Allocation model.
In short, I have data that comes from a $K$-dimensional Dirichlet-Multinomial distribution, i.e. $$\theta_d \sim \text{Dirichlet}(\boldsymbol{\alpha}), \quad \forall d = 1,\ldots,D$$ $$z_{d,n} \sim \text{Categorical}(\theta_d), \quad \forall n = 1,\ldots,N_d$$ where $z_{d,n}\in[1,K]$, $\theta_d$ is a $K$ dimensional probability vector, and in principle $\boldsymbol{\alpha}=(\alpha_1,\ldots,\alpha_K)$ is a $K$ dimensional vector of positive numbers. Additionally, I have a hyper-prior on $\boldsymbol{\alpha}$: $$\alpha_k \sim \text{Gamma}(a,b), \quad \forall k=1,\ldots,K$$ Oddly enough, I had no problem finding a way to estimate $\boldsymbol{\alpha}$ in such case, i.e. finding maximum of $p(\boldsymbol{\alpha}|\mathcal{Z})$ - it is done through fixed-point iteration, as presented in Wallach (2006), p. 39., where the new value is estimated as follows: $${\alpha_K}^*=\alpha_k \frac{\sum_{d=1}^D \Big( \Psi(N_{k|d} + \alpha_k) - \Psi(\alpha_k) \Big) + a}{\sum_{d=1}^D \Big( \Psi(N_d + \sum_{k=1}^K\alpha_k) - \Psi(\sum_{k=1}^K\alpha_k) \Big) - \frac{1}{b}}$$ where $N_{k|d}$ is the number of $z_{d,n}$'s in $d$ that where assigned to $k$ and $\Psi$ is the digamma function.
What I need, is a method for finding an analogous estimate but under the assumption that all elements of $\boldsymbol{\alpha}$ are equal. This seems like a simpler case and in LDA is often assumed for the topic-term distributions. Nevertheless, for the life of me I cannot find it anywhere, and I am unable to derive it myself - intuitively, averaging $\alpha_k$'s from the above method makes sense, but I am not sure and perhaps there is a more computationally efficient algorithm.
