2
$\begingroup$

So if we have a Poisson distribution with a rate lambda we know that the prior is a gamma with alpha,beta. But suppose we didn't know that the prior was a gamma. How would we do the derivation please?

I think it's pretty straightforward and I'm just not seeing it.

$\endgroup$
2
  • $\begingroup$ Your question doesn't make sense - priors for a parameter don't generally come from examining the density of the model for the data. Are you asking about a particular kind of prior, like a Jeffreys' prior, or maybe just about conjugate priors, where you would look at the form of the probability function? Please clarify. $\endgroup$
    – Glen_b
    Nov 2, 2014 at 0:32
  • $\begingroup$ Conjugate prior please. I was thinking about if you had a prob mass dist which did not appear on one of the standard tables $\endgroup$ Nov 2, 2014 at 1:16

1 Answer 1

1
$\begingroup$

Consider the posterior as prior$\times$likelihood and we want the posterior to be of the same form as the prior, so the likelihood must combine with the prior in a way that leaves the prior unchanged in form, apart from the values of the parameters.

In the case of the Poisson, we can do this one by inspection.

Recall that the likelihood is a function of the parameter, not the data (the data are fixed - we condition on them):

$\mathcal{L}(\lambda;\mathbf{x})\propto \lambda^{S} \exp(-n\lambda)\propto \lambda^{(S+1)-1} \exp(-n\lambda)$

(where $S$ is an obvious function of the data). We can immediately see that the likelihood is (up to scaling constants) of the form of a gamma density in $\lambda$.

This suggests to us that perhaps a gamma density could be a conjugate prior, since that also consists of a power of $\lambda$ and $\exp(-k\lambda)$, both of which will combine when we take the product:

$p(\theta|\mathbf{x})\propto \mathcal{L}(\lambda;\mathbf{x})\propto \lambda^{S} \exp(-n\lambda)\cdot \lambda^{\alpha-1}\exp(-\lambda \theta)$

$\hspace{1.4cm}\propto \lambda^{(S+\alpha)-1} e^{-\lambda(n+\theta)}$

Often this is all that it takes - looking at the likelihood as a function of the parameter and (in simple cases) recognizing the form of density where products of such densities are scaled densities of the same kind (as is the case with normal or gammas). In other cases, you will instead need to recognize that there's some density which the likelihood combines with in a product which will also be of the same form as the prior (even though the likelihood itself perhaps isn't in that form).

For another example, see this discussion of a conjugate prior for the Laplace (in the sense that the posterior is in the same form as the prior).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.