# Exponential likelihood + Exponential prior =?

Following Kevin Murphy's MLAPP and I am having troubles with Exercise 3.11. The exercise happens in parts, and I am hoping to get help with multiple parts.

A lifetime $$X$$ of a machine is modeled by an exponential distribution with unknown parameter $$\theta$$. The likelihood is $$p(x \mid \theta) = \theta e^{-\theta x}$$

Assume an expert believes $$\theta$$ should have a prior distribution that is also exponential $$p(\theta) = \text{Expon}(\theta \mid \lambda) = \text{Gamma}(\theta \mid 1, \lambda) = \lambda e^{-\lambda \theta}$$

Note: $$\mathcal{D} = \{ x_1, \dots, x_N \}$$ is the data.

## 1. What is the posterior $$p(\theta \mid \mathcal{D}, \lambda)$$?

I get \begin{align} p(\theta \mid \mathcal{D}, \lambda) &= p(\theta \mid \lambda) p(\mathcal{D} \mid \theta) \\ &= \lambda e^{-\lambda \theta} \prod_{i=1}^N \theta e^{-\theta x_i} \\ &= \lambda \theta^N e^{-\theta (\lambda + \sum_{i=1}^N x_i)} \end{align}

which doesn't look like any distribution that I know of.

## 2. Is the exponential prior conjugate to the exponential likelihood?

No. The posterior is not an exponential. But I'm not sure if I computed the posterior correctly.

I assume that I'm doing something wrong, because I'm later asked about the posterior mean, which is difficult to do by integration (integration by parts and extracting a term that depends upon the Gamma function).

## 1 Answer

Hints:

1. Your posterior density is only proportionate to that expression, not equal

2. Both your prior density and posterior density (when sorted out) are Gamma distributions

3. The conjugate family for the rate of an exponential distribution are Gamma distributions

• To clarify: "The conjugate family for the rate of an exponential distribution" means that the conjugate for p(theta | D, lambda) should be a gamma function? – Alex Van de Kleut May 7 '20 at 17:21
• I might not use your notation because I think of the prior as a Gamma distribution rather than an exponential distribution (in your example with shape parameter $1$ and rate parameter $\lambda$) and I might prefer $\pi$ for the density of $\theta$ rather than $p$, but essentially yes. – Henry May 7 '20 at 17:43