# Show posterior is proper for this poisson linear model

This question is 3.12 in Andrew Gelman's Bayesian Data Analysis 3rd edition.

Let $y_i|\alpha,\beta \overset{iid}{\sim} \text{Poisson}$ with mean $\alpha+\beta t_i$.

Find a prior distribution that is "noninformative" such that the posterior $\alpha,\beta|\mathbf{y}$ is a proper distribution.

MY ATTEMPT

\begin{align*} p(\alpha,\beta|\mathbf{y}) &\propto p(\mathbf{y}|\alpha,\beta)p(\alpha,\beta)\\ &= \prod_{i=1}^n \dfrac{e^{-(\alpha+\beta t_i)}(\alpha+\beta t_i)^{y_i}}{y_i!} p(\alpha,\beta) \end{align*}

My first attempt is a flat prior $p(\alpha,\beta)\propto 1$ however I can't think of how to integrate the posterior. My next attempt is Jeffrey's prior \begin{align*} \ell(\alpha,\beta) &= -\sum_{i=1}^n (\alpha+\beta t_i) + \sum_{i=1}^n y_i \log{(\alpha+\beta t_i)} - \sum_{i=1}^n \log{y_i!}\\ \partial\ell/\partial\alpha &= -n + \sum_{i=1}^n \dfrac{y_i}{\alpha+\beta t_i}\\ \partial\ell/\partial\beta &= -\sum_{i=1}^n t_i + \sum_{i=1}^n \dfrac{y_it_i} {\alpha+\beta t_i}\\ \partial^2\ell/\partial\alpha^2 &= - \sum_{i=1}^n \dfrac{y_i}{(\alpha+\beta t_i)^2}\\ \partial^2\ell/\partial\beta^2 &= - \sum_{i=1}^n \dfrac{y_it_i^2}{(\alpha+\beta t_i)^2}\\ \partial^2\ell/\partial\alpha\partial\beta &= - \sum_{i=1}^n \dfrac{y_it_i}{(\alpha+\beta t_i)^2}\\ I(\alpha,\beta) &= \text{E}\left[ \dfrac{-\partial^2\ell}{\partial(\alpha,\beta)\partial(\alpha,\beta)}'\right]\\ &= \begin{pmatrix} \sum_{i=1}^n \dfrac{1}{\alpha+\beta t_i} & \sum_{i=1}^n \dfrac{t_i}{\alpha+\beta t_i} \\ \sum_{i=1}^n \dfrac{t_i}{\alpha+\beta t_i} & \sum_{i=1}^n \dfrac{t_i^2}{\alpha+\beta t_i} \end{pmatrix}\\ p(\alpha,\beta) &\propto \sqrt{|I(\alpha,\beta)|}\\ &= \sqrt{ \sum_{i=1}^n \dfrac{1}{\alpha+\beta t_i}\sum_{i=1}^n \dfrac{t_i^2}{\alpha+\beta t_i} - \left(\sum_{i=1}^n \dfrac{t_i}{\alpha+\beta t_i} \right)^2} \end{align*}

But this doesn't help me integrate the posterior either. Any hints or ideas?

I recall a user @cyan did all of the BDA homework problems a few years ago. Summoning him.

• @ summons don't work in a question or answer, and they only work in comments if the user has been involved in that question in some way (commenting or answering). I'm not sure I'd call a flat prior on the half-line "noninformative". (In fact ... how does Gelman define noninformative?) Commented Feb 18, 2015 at 2:07

Let $\theta=(\alpha,\beta)$ and $p(\theta)$ be a proper prior. Let $p(\theta|y)$ be the un-normalized posterior. We want to show that $$\int p(\theta|y)d\theta \propto \int p(y|\theta)p(\theta)d\theta = p(y) < \infty$$ Note that $y$ is a discrete random variable. Let $\mathcal{Y}$ denote its support \begin{align*} p(y) &< \sum_{y\in\mathcal{Y}} p(y)\\ &= \sum_{y\in\mathcal{Y}} \int p(y|\theta)p(\theta)d\theta\\ &= \int \sum_{y\in\mathcal{Y}} p(y|\theta)p(\theta)d\theta\\ &= \int p(\theta)d\theta\\ &= 1 \end{align*} Therefore $p(y)$ is finite and therefore the posterior is proper.
• When $y$ is a random variable, "$p(y)$" makes no sense: probabilities are associated with events, which are sets of possible values. You do seem to calculate with $y$ as if it were a particular value. But since uncountably many values are possible with a Poisson distribution, we cannot conclude from the result $p(y)\le 1$ that the posterior is proper!
• let $p(y)$ be the pmf Commented Feb 19, 2015 at 19:06
• That's what I had surmised: but it's immediate that $p(y)\le 1$ for any pmf, so you haven't shown anything new, and in fact all you can conclude is that $\sum_{y\in\mathcal Y}p(y) \le |\mathcal{Y}| = \infty$, which does no good.
• You are right sorry, $p(y|\theta)$ is the pmf, $p(y)$ is the marginal likelihood $p(y)=\int p(y|\theta)p(\theta)d\theta$ Commented Feb 19, 2015 at 19:12