I observe independent, Poisson-distributed data $ D = \{x_1, ... x_n \} $ with mean parameter $ \mu $, i.e., $$x_i\stackrel{\text{iid}}{\sim}\mathcal{P}(\mu)$$ Over $ \mu $ I assume $ Gamma(\alpha_0, \beta_0) $ as a prior (where $ \alpha_0 $ and $ \beta_0 $ are some given hyperparameters). The posterior distribution is $$\mu|D \sim Gamma\left(a_0 + \sum\limits_{n=1}^N x_n, \beta_0 + N\right) $$ I am interested in the mean and variance of the predictive distribution $p(x_{N+1} | D)$.

I know it's possible to show that the predictive distribution is a negative binomial, but, since I am only interested in mean and variance there should be a much quicker way by using identities, the law of total expectation and the law of total variance. However, I get confused when trying to apply them here.

The posterior predictive distribution for $ x_{N+1} $ is (leaving out conditioning on hyperparameters): $$ p(x_{N+1}|D) = \int_{\mu} p(x_{N+1}|\mu) \, p(\mu|D) \operatorname{d}\!\mu $$

That is equivalent to the expectation:

$$ p(x_{N+1}|D) = \mathbb{E}_{\mu|D}\Big[p(x_{N+1}|\mu)\Big] $$

The mean of the posterior distribution then is:

$$ \mathbb{E}_{x_{N+1}|D} \Big[\mathbb{E}_{\mu|D}\Big[p(x_{N+1}|\mu)\Big]\Big]$$

which is an iterated expectation to which I would like to apply the law of total expectation.

The law states: $$ \mathbb{E}_X[X] = \mathbb{E}_Y[\mathbb{E}_{X|Y}[X|Y]] $$

Looking at the indices however, I don't know how to proceed from here: That is I cannot map my indices properly to the ones of the law of total expectation.

  • 1
    $\begingroup$ Are you sure that the law of total expectation even applies here? I agree that the theorem does not seem applicable based on the "plug-in" approach. $\endgroup$
    – user44764
    Dec 16, 2014 at 0:08
  • 1
    $\begingroup$ This is a negative binomial distribution ; see appendices B & C here orbi.ulg.ac.be/bitstream/2268/77820/1/… $\endgroup$ Dec 31, 2014 at 10:44

1 Answer 1


While writing$$p(x_{N+1}|D)=\mathbb{E}_{\mu|D}[p(x_{N+1}|\mu)]$$is formally correct, this leads to confusion when applying the Law of Total Expectation, in that the formula $$\mathbb{E}_{x_{N+1}|D} \Big[\mathbb{E}_{\mu|D}\Big[p(x_{N+1}|\mu)\Big]\Big]$$ does not make sense.

What you want to find is $\mathbb{E}_{x_{N+1}|D}[X_{N+1}]$ and $\text{var}_{x_{N+1}|D}[X_{N+1}]$. So, if you apply the Law of Total Expectation, you get $$\mathbb{E}_{x_{N+1}|D}[X_{N+1}]=\mathbb{E}_{\mu|D}\left\{\mathbb{E}_{x_{N+1}|\mu,D}[X_{N+1}]\right\}=\mathbb{E}_{\mu|D}\left\{\mu\right\}=\frac{a_0 + \sum_{n=1}^N x_n}{\beta_0 + N}\,.$$

Similarly, for the variance, \begin{align*} \text{var}_{x_{N+1}|D}[X_{N+1}] &= \mathbb{E}_{\mu|D}\left\{\text{var}_{x_{N+1}|\mu,D}[X_{N+1}]\right\}+\text{var}_{\mu|D}[\mathbb{E}_{x_{N+1}|\mu,D}\left\{X_{N+1}\right\}]\\ &= \mathbb{E}_{\mu|D}\left\{\mu\right\}+\text{var}_{\mu|D}[\mu]\\ &= \frac{a_0 + \sum_{n=1}^N x_n}{\beta_0 + N}+\frac{a_0 + \sum_{n=1}^N x_n}{(\beta_0 + N)^2}\\ &= \frac{a_0 + \sum_{n=1}^N x_n}{\beta_0 + N}\,\frac{\beta_0 + N+1}{\beta_0 + N} \end{align*}


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.