This question is 3.12 in Andrew Gelman's Bayesian Data Analysis 3rd edition.

Let $y_i|\alpha,\beta \overset{iid}{\sim} \text{Poisson}$ with mean $\alpha+\beta t_i$.

Find a prior distribution that is "noninformative" such that the posterior $\alpha,\beta|\mathbf{y}$ is a proper distribution.


$$\begin{align*} p(\alpha,\beta|\mathbf{y}) &\propto p(\mathbf{y}|\alpha,\beta)p(\alpha,\beta)\\ &= \prod_{i=1}^n \dfrac{e^{-(\alpha+\beta t_i)}(\alpha+\beta t_i)^{y_i}}{y_i!} p(\alpha,\beta) \end{align*}$$

My first attempt is a flat prior $p(\alpha,\beta)\propto 1$ however I can't think of how to integrate the posterior. My next attempt is Jeffrey's prior $$\begin{align*} \ell(\alpha,\beta) &= -\sum_{i=1}^n (\alpha+\beta t_i) + \sum_{i=1}^n y_i \log{(\alpha+\beta t_i)} - \sum_{i=1}^n \log{y_i!}\\ \partial\ell/\partial\alpha &= -n + \sum_{i=1}^n \dfrac{y_i}{\alpha+\beta t_i}\\ \partial\ell/\partial\beta &= -\sum_{i=1}^n t_i + \sum_{i=1}^n \dfrac{y_it_i} {\alpha+\beta t_i}\\ \partial^2\ell/\partial\alpha^2 &= - \sum_{i=1}^n \dfrac{y_i}{(\alpha+\beta t_i)^2}\\ \partial^2\ell/\partial\beta^2 &= - \sum_{i=1}^n \dfrac{y_it_i^2}{(\alpha+\beta t_i)^2}\\ \partial^2\ell/\partial\alpha\partial\beta &= - \sum_{i=1}^n \dfrac{y_it_i}{(\alpha+\beta t_i)^2}\\ I(\alpha,\beta) &= \text{E}\left[ \dfrac{-\partial^2\ell}{\partial(\alpha,\beta)\partial(\alpha,\beta)}'\right]\\ &= \begin{pmatrix} \sum_{i=1}^n \dfrac{1}{\alpha+\beta t_i} & \sum_{i=1}^n \dfrac{t_i}{\alpha+\beta t_i} \\ \sum_{i=1}^n \dfrac{t_i}{\alpha+\beta t_i} & \sum_{i=1}^n \dfrac{t_i^2}{\alpha+\beta t_i} \end{pmatrix}\\ p(\alpha,\beta) &\propto \sqrt{|I(\alpha,\beta)|}\\ &= \sqrt{ \sum_{i=1}^n \dfrac{1}{\alpha+\beta t_i}\sum_{i=1}^n \dfrac{t_i^2}{\alpha+\beta t_i} - \left(\sum_{i=1}^n \dfrac{t_i}{\alpha+\beta t_i} \right)^2} \end{align*}$$

But this doesn't help me integrate the posterior either. Any hints or ideas?

I recall a user @cyan did all of the BDA homework problems a few years ago. Summoning him.

  • 1
    $\begingroup$ @ summons don't work in a question or answer, and they only work in comments if the user has been involved in that question in some way (commenting or answering). I'm not sure I'd call a flat prior on the half-line "noninformative". (In fact ... how does Gelman define noninformative?) $\endgroup$
    – Glen_b
    Commented Feb 18, 2015 at 2:07

1 Answer 1


So what I think the answer wanted was to show that if sampling distribution is proper, and you use a proper prior distribution, you end up with a proper posterior. A quick proof:

Let $\theta=(\alpha,\beta)$ and $p(\theta)$ be a proper prior. Let $p(\theta|y)$ be the un-normalized posterior. We want to show that $$\int p(\theta|y)d\theta \propto \int p(y|\theta)p(\theta)d\theta = p(y) < \infty$$ Note that $y$ is a discrete random variable. Let $\mathcal{Y}$ denote its support \begin{align*} p(y) &< \sum_{y\in\mathcal{Y}} p(y)\\ &= \sum_{y\in\mathcal{Y}} \int p(y|\theta)p(\theta)d\theta\\ &= \int \sum_{y\in\mathcal{Y}} p(y|\theta)p(\theta)d\theta\\ &= \int p(\theta)d\theta\\ &= 1 \end{align*} Therefore $p(y)$ is finite and therefore the posterior is proper.

  • $\begingroup$ When $y$ is a random variable, "$p(y)$" makes no sense: probabilities are associated with events, which are sets of possible values. You do seem to calculate with $y$ as if it were a particular value. But since uncountably many values are possible with a Poisson distribution, we cannot conclude from the result $p(y)\le 1$ that the posterior is proper! $\endgroup$
    – whuber
    Commented Feb 19, 2015 at 19:03
  • $\begingroup$ let $p(y)$ be the pmf $\endgroup$
    – bdeonovic
    Commented Feb 19, 2015 at 19:06
  • $\begingroup$ That's what I had surmised: but it's immediate that $p(y)\le 1$ for any pmf, so you haven't shown anything new, and in fact all you can conclude is that $\sum_{y\in\mathcal Y}p(y) \le |\mathcal{Y}| = \infty$, which does no good. $\endgroup$
    – whuber
    Commented Feb 19, 2015 at 19:08
  • $\begingroup$ You are right sorry, $p(y|\theta)$ is the pmf, $p(y)$ is the marginal likelihood $p(y)=\int p(y|\theta)p(\theta)d\theta$ $\endgroup$
    – bdeonovic
    Commented Feb 19, 2015 at 19:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.