What is the posterior distribution of θ? Is the Gamma a conjugate prior for an exponential likelihood?

Question

A manufacturer is interested in the time to failure of his batteries.

Suppose the time to failure of the batteries has an exponential distribution:

$$p(x│\theta)=\theta\exp(-\theta x)$$

Note that the mean of this population is $1/\theta.$

The manufacturer is interested in estimating $\theta.$ Suppose that $n $ batteries are randomly selected and let their failure times be $x_1, x_2,\ldots, x_n.$

Suppose we use a gamma prior for $\theta$ $$p(\theta│\alpha,\beta)=\beta^\alpha/(\Gamma(\alpha)) \theta^{(\alpha-1) }\exp(-\beta\theta)$$

a) What is the posterior distribution of $\theta$? Is the Gamma a conjugate prior for an exponential likelihood?

b) Suppose an expert believes that the mean time to failure is $120$ hours. Furthermore, he claims that he is $95\%$ sure that the mean time to failure is between $100$ and $150$ hours. What would be a suitable choice for $\alpha$ and $\beta$ in the prior?

Answer 1

Gamma distribution is indeed a conjugate prior for an exponential likelihood, because they share the same kernel for the parameter $\theta$, i.e.

The Gamma prior for $\theta$ has

$p(\theta;a,b)\propto \theta^{a-1}e^{-b\theta}$

whereas the exponential likelihood for all the independent identically distributed observations $x_{1}, x_{2},..., x_{n}$ is

$l(\theta;x_{1}, x_{2},..., x_{n}) = \theta e^{-\theta x_{1}}\theta e^{-\theta x_{2}}...\theta e^{-\theta x_{n}} = \theta^{n} e^{-\theta \sum_{i=1}^{n}x_{i}}$

You can observe that both prior and likelihood have the same kernel, i.e. $\theta$ in a power multiplied by exponential of $\theta$, this makes it possible to multiply $p(\theta;a,b)$ with $l(\theta ;x_{1}, x_{2},..., x_{n})$ and get a standard output.

Hence, the posterior will be

$p(\theta|x_{1}, x_{2},..., x_{n}) \propto l(\theta;x_{1}, x_{2},...,x_{n}) p(\theta; a,b) = \theta^{n} e^{-\theta\sum_{i=1}^{n}}\theta^{a-1}e^{-b\theta}=\theta^{n+a-1}e^{-\theta(b+\sum_{i=1}^{n}x_{i})} $

This kernel might remind you the kernel of a Gamma distribution, so $\theta\sim Gamma(a+n,b+\sum_{i=1}^{n}x_{i})$

---

Now if you know that apriori the mean of $\theta$ is 120, you can translate it as $\mathbb{E}[\theta]_{p(\theta)} = \frac{a}{b}=120$ which is the expected value of $\theta$ under the prior $p(\theta;a,b)$.

So, you have one equation $a=120b \ \ \ (*)$

Then you also, have the information that $\mathbb{P}(100\leq \theta \leq 150) = 0.95$

$ \Rrightarrow \mathbb{P}(\theta \leq 150)-\mathbb{P}(\theta \leq 100) = 0.95 $

$\Rrightarrow \frac{\gamma(a,b 150)}{\Gamma(a)} -\frac{\gamma(a,b 100)}{\Gamma(a)}=0.95 \Rightarrow \frac{\gamma(a,50b)}{\Gamma(a)} = 0.95 \ \ \ (**)$

Now plugging in the $(*)$ into $(**)$ you get

$\frac{\gamma(120b,50b)}{\Gamma(120b)} = 0.95 \ \ (***)$ which depends only on $b$.

So, what you can do is define a grid of values for $b$ and calculate $(***)$ with the use of $R$ language or any other language, and check if this equality holds $\frac{\gamma(120b,50b)}{\Gamma(120b)} = 0.95$ and denote as $b^{*}$ the $b$ for which you found that the equality holds, then calculate as $a^{*}=120b^{*}$. And those $a^{*},b^{*}$ are the parameters that reflect your prior beliefs.