This question is 3.12 in Andrew Gelman's Bayesian Data Analysis 3rd edition.

Let $y_i|\alpha,\beta \overset{iid}{\sim} \text{Poisson}$ with mean $\alpha+\beta t_i$.

Find a prior distribution that is "noninformative" such that the posterior $\alpha,\beta|\mathbf{y}$ is a proper distribution.


$$\begin{align*} p(\alpha,\beta|\mathbf{y}) &\propto p(\mathbf{y}|\alpha,\beta)p(\alpha,\beta)\\ &= \prod_{i=1}^n \dfrac{e^{-(\alpha+\beta t_i)}(\alpha+\beta t_i)^{y_i}}{y_i!} p(\alpha,\beta) \end{align*}$$

My first attempt is a flat prior $p(\alpha,\beta)\propto 1$ however I can't think of how to integrate the posterior. My next attempt is Jeffrey's prior $$\begin{align*} \ell(\alpha,\beta) &= -\sum_{i=1}^n (\alpha+\beta t_i) + \sum_{i=1}^n y_i \log{(\alpha+\beta t_i)} - \sum_{i=1}^n \log{y_i!}\\ \partial\ell/\partial\alpha &= -n + \sum_{i=1}^n \dfrac{y_i}{\alpha+\beta t_i}\\ \partial\ell/\partial\beta &= -\sum_{i=1}^n t_i + \sum_{i=1}^n \dfrac{y_it_i} {\alpha+\beta t_i}\\ \partial^2\ell/\partial\alpha^2 &= - \sum_{i=1}^n \dfrac{y_i}{(\alpha+\beta t_i)^2}\\ \partial^2\ell/\partial\beta^2 &= - \sum_{i=1}^n \dfrac{y_it_i^2}{(\alpha+\beta t_i)^2}\\ \partial^2\ell/\partial\alpha\partial\beta &= - \sum_{i=1}^n \dfrac{y_it_i}{(\alpha+\beta t_i)^2}\\ I(\alpha,\beta) &= \text{E}\left[ \dfrac{-\partial^2\ell}{\partial(\alpha,\beta)\partial(\alpha,\beta)}'\right]\\ &= \begin{pmatrix} \sum_{i=1}^n \dfrac{1}{\alpha+\beta t_i} & \sum_{i=1}^n \dfrac{t_i}{\alpha+\beta t_i} \\ \sum_{i=1}^n \dfrac{t_i}{\alpha+\beta t_i} & \sum_{i=1}^n \dfrac{t_i^2}{\alpha+\beta t_i} \end{pmatrix}\\ p(\alpha,\beta) &\propto \sqrt{|I(\alpha,\beta)|}\\ &= \sqrt{ \sum_{i=1}^n \dfrac{1}{\alpha+\beta t_i}\sum_{i=1}^n \dfrac{t_i^2}{\alpha+\beta t_i} - \left(\sum_{i=1}^n \dfrac{t_i}{\alpha+\beta t_i} \right)^2} \end{align*}$$

But this doesn't help me integrate the posterior either. Any hints or ideas?

I recall a user @cyan did all of the BDA homework problems a few years ago. Summoning him.

  • 1
    $\begingroup$ @ summons don't work in a question or answer, and they only work in comments if the user has been involved in that question in some way (commenting or answering). I'm not sure I'd call a flat prior on the half-line "noninformative". (In fact ... how does Gelman define noninformative?) $\endgroup$
    – Glen_b
    Feb 18 '15 at 2:07

So what I think the answer wanted was to show that if sampling distribution is proper, and you use a proper prior distribution, you end up with a proper posterior. A quick proof:

Let $\theta=(\alpha,\beta)$ and $p(\theta)$ be a proper prior. Let $p(\theta|y)$ be the un-normalized posterior. We want to show that $$\int p(\theta|y)d\theta \propto \int p(y|\theta)p(\theta)d\theta = p(y) < \infty$$ Note that $y$ is a discrete random variable. Let $\mathcal{Y}$ denote its support \begin{align*} p(y) &< \sum_{y\in\mathcal{Y}} p(y)\\ &= \sum_{y\in\mathcal{Y}} \int p(y|\theta)p(\theta)d\theta\\ &= \int \sum_{y\in\mathcal{Y}} p(y|\theta)p(\theta)d\theta\\ &= \int p(\theta)d\theta\\ &= 1 \end{align*} Therefore $p(y)$ is finite and therefore the posterior is proper.

  • $\begingroup$ When $y$ is a random variable, "$p(y)$" makes no sense: probabilities are associated with events, which are sets of possible values. You do seem to calculate with $y$ as if it were a particular value. But since uncountably many values are possible with a Poisson distribution, we cannot conclude from the result $p(y)\le 1$ that the posterior is proper! $\endgroup$
    – whuber
    Feb 19 '15 at 19:03
  • $\begingroup$ let $p(y)$ be the pmf $\endgroup$
    – bdeonovic
    Feb 19 '15 at 19:06
  • $\begingroup$ That's what I had surmised: but it's immediate that $p(y)\le 1$ for any pmf, so you haven't shown anything new, and in fact all you can conclude is that $\sum_{y\in\mathcal Y}p(y) \le |\mathcal{Y}| = \infty$, which does no good. $\endgroup$
    – whuber
    Feb 19 '15 at 19:08
  • $\begingroup$ You are right sorry, $p(y|\theta)$ is the pmf, $p(y)$ is the marginal likelihood $p(y)=\int p(y|\theta)p(\theta)d\theta$ $\endgroup$
    – bdeonovic
    Feb 19 '15 at 19:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.