What is the bayesian uninformative exponential prior?

Question

I am looking to model a poisson process $X \sim Poisson(\lambda)$ and infer $\lambda$ using bayesian inference from $X$ e.g. $[0,0,0,2,1,0,0,2,1]$ could be my dataset. The data comes from the count of something occurring in a day.

I am considering $Exponential(\alpha)$ as my prior as it can represent positive real numbers, and I am wondering how to pick $\alpha$. If it is not possible to do so in an uninformative way, I would also like to know if there are methods to sensibly pick $\alpha$ or possibly whether there is a more suitable prior distribution.

The output would be a distribution around $\lambda$

Answer 1

The conjugate prior for a Poisson likelihood is the Gamma distribution, conveniently described with a shape parameter and a rate parameter (the inverse of a scale parameter).

Your choice of an exponential distribution would correspond to a prior shape parameter of $1$, at which point you would need to choose a prior rate parameter, inserting information into your prior.

An alternative approach might be to choose a prior shape parameter of $0$ and a prior rate parameter of $0$, which is improper and largely meaningless in itself but would have the result that after observing $x_1,x_2,\ldots,x_n$ your posterior distribution would have a shape parameter of $\sum\limits_{i=1}^n x_i$ and a rate parameter of $n$. Providing that at least one of your observations was non-zero, this could be a fairly sensible posterior distribution with mean the intuitive $\frac{\sum x_i}{n}$, and easy to work with and update further; you can see how little prior information is involved.

In your example, your Gamma posterior distribution for $\lambda$ would then have shape parameter $6$ and rate parameter $9$ under this alternative approach; compare this with starting with an exponential prior where the posterior would have shape parameter $7$ and rate parameter $9$ plus whatever rate you had chosen initially for your exponential.