I am reading Escobar&West paper and in particular am interested in their Gibbs sampler for the concentration parameter of Dirichlet Process (eq 13, eq 14). Given this,

$$p(\alpha,\eta|k)\propto p(\alpha)\alpha^{k-1}(\alpha+n)\eta^\alpha(1-\eta)^{n-1}$$

If we replace $p(\alpha)$, with it's $Gamma(a,b)$ distribution replacement, we get,

$$p(\alpha,\eta|k)\propto \large(\frac{1}{\Gamma(a)b^a}\alpha^{a-1}e^{\frac{-\alpha}{b}}\large)\alpha^{k-1}(\alpha+n)\eta^\alpha(1-\eta)^{n-1}$$

which can be re-written as,

$$p(\alpha,\eta|k)\propto \frac{1}{\Gamma(a)b^a}e^{\frac{-\alpha}{b}}\alpha^{a+k-2}(\alpha+n)e^{\log\eta^\alpha}(1-\eta)^{n-1}$$

and can be simplified again as,

$$p(\alpha,\eta|k)\propto \frac{1}{\Gamma(a)b^a}\alpha^{a+k-2}(\alpha+n)e^{(-\frac{\alpha}{b}+\alpha\log\eta)}(1-\eta)^{n-1}$$

and to convert this to $p(\alpha|\eta,k)$, we must take the integral,

$$p(\alpha|\eta,k)\propto \frac{1}{\Gamma(a)b^a}\alpha^{a+k-2}(\alpha+n)\int e^{(-\frac{\alpha}{b}+\alpha\log\eta)}(1-\eta)^{n-1}d\eta$$

but I don't understand how from that we can get to the following relation mentioned in the paper,

$$p(\alpha|\eta,k)\propto \alpha^{a+k-2}(\alpha+n)e^{\alpha(-b+\log(\eta))}$$

Assuming that they dropped redundant terms that do not depend on $\alpha$, for example, $\frac{1}{\Gamma(a)b^a}$ and $(1-\eta)^{n-1}$, still it is not clear how they computed the integral. Also, the power of $e$ must be ${\alpha(-\frac{1}{b}+\log(\eta))}$ unless I am skipping a step.


You are making an error when taking the integral. Indeed,

$$p(\alpha|\eta,k) \neq p(\alpha|k) = \int p(\alpha, \eta|k) d\eta.$$

Instead of integrating, you ought to use Bayes' rule;

$$ p(\alpha|\eta, k) \propto p(\alpha, \eta|k) $$

where the proportionality constant $1/p(\eta|k)$ does not need to be computed since it is constant with respect to $\alpha$. Indeed, we can remove more factors which do not depend on $\alpha$ in your expression of $p(\alpha, \eta|k)$, which yields an answer similar to the one the authors arrive at. (There is a small difference which leads me to suspect that there is a transcription error somewhere.)

| cite | improve this answer | |
  • $\begingroup$ Nice catch! The error you are referring to, is it the power of e which I mentioned in the end? $\endgroup$ – user3639557 Jul 27 '16 at 10:23
  • $\begingroup$ @Martin Lindfors I agree with your main point. But the observation that the conditional distribution is proportional to the joint distribution is not Bayes' rule. Bayes' rule is about the relation between two conditional distributions. Second, I suspect the OP and the article are using two different parameterizations for the gamma distribution, which leads to 1/b in place of b. $\endgroup$ – mef Jul 27 '16 at 16:41
  • $\begingroup$ @user3639557 Correct. $\endgroup$ – Martin L Jul 31 '16 at 21:02
  • $\begingroup$ @mef That is one version of Bayes' rule since $p(\alpha, \eta|k) = p(\eta|\alpha, k) p(\alpha|k)$. $\endgroup$ – Martin L Jul 31 '16 at 21:03
  • $\begingroup$ @mef I don't think I am using a different Gamma definition compared to the paper (we both use shape and scale). On contrary, I think if the paper changes the Gamma definition to be based on (rate and shape), the math will be correct. But the math in the paper as-is seems incorrect. $\endgroup$ – user3639557 Aug 2 '16 at 9:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.