We know that if a random variable say x ~ Gamma(a, b), then its probability density function is $ \propto x^{a-1} exp^{-bx}$.

In a Bayesian hierarchical model, for example

$Z_1, \cdots, Z_n |\theta \sim iid \; Gamma(r, \theta)$, (r known)$ ;

$\theta \sim \; Gamma(a, b)$,

the full conditional distribution of $\theta$ is

$p(\theta | Z_{1:n}) \propto p(\theta, Z_{1:n}) \propto p(Z_{1:n}|\theta) \times p(\theta)$,

by $Z_{1:n} | \theta \sim iid \; Gamma(r, \theta)$, I think $p(Z_{1:n}|\theta)$ shall be $\Pi_i Z_i^{r-1} exp^{- \theta Z_i}$, but the reading material suggests

$p(Z_{1:n}|\theta) = \Pi_i \theta^{r} exp^{-\theta Z_i}$,

I'm not sure I fully understand why it writes $\theta^{r}$ instead of $Z_i^{r-1}$ in front of $exp^{-\theta Z_i}$.

Can anyone explain? Thanks.


1 Answer 1


The full joint density of $Z_{1:n}$ conditional on $\theta$ is $$ p(z_{1:n}|\theta)= \prod_{i=1}^n \frac{\theta^r}{\Gamma(r)} z_i^{r-1} \exp(-\theta z_i) \,. $$ If we consider this as a function of $z_{1:n}$, the term $\theta^{r}/\Gamma(r)$ is just a normalising constant and can be dropped.

As you point out, the conditional distribution of $\theta$ given $z_{1:n}$ (i.e. the posterior of $\theta$ given the observed sample) is $$ p(\theta|z_{1:n}) \propto p(\theta) p(z_{1:n}|\theta) = p(\theta) \prod_{i=1}^n \frac{\theta^r}{\Gamma(r)} z_i^{r-1} \exp(-\theta z_i) $$ This is a function of $\theta$, so we need to keep the terms involving $\theta$ (such as $\theta^{r}$), but we can drop the other terms (such as $z_i^{r-1}$). We can thus write $$ p(\theta|z_{1:n}) \propto p(\theta) \prod_{i=1}^n \theta^r \exp(-\theta z_i) \propto \theta^{a-1} \exp(-b\theta)\,\, \theta^{nr} \exp(-\theta n \bar{z}) $$ which we identify as another Gamma density.

  • 1
    $\begingroup$ Thanks for your clear explanation! $\endgroup$
    – Alison
    Commented Feb 20, 2023 at 21:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.