3
$\begingroup$

Following Kevin Murphy's MLAPP and I am having troubles with Exercise 3.11. The exercise happens in parts, and I am hoping to get help with multiple parts.


A lifetime $X$ of a machine is modeled by an exponential distribution with unknown parameter $\theta$. The likelihood is $$ p(x \mid \theta) = \theta e^{-\theta x} $$

Assume an expert believes $\theta$ should have a prior distribution that is also exponential $$ p(\theta) = \text{Expon}(\theta \mid \lambda) = \text{Gamma}(\theta \mid 1, \lambda) = \lambda e^{-\lambda \theta} $$

Note: $\mathcal{D} = \{ x_1, \dots, x_N \}$ is the data.


1. What is the posterior $p(\theta \mid \mathcal{D}, \lambda)$?

I get $$ \begin{align} p(\theta \mid \mathcal{D}, \lambda) &= p(\theta \mid \lambda) p(\mathcal{D} \mid \theta) \\ &= \lambda e^{-\lambda \theta} \prod_{i=1}^N \theta e^{-\theta x_i} \\ &= \lambda \theta^N e^{-\theta (\lambda + \sum_{i=1}^N x_i)} \end{align} $$

which doesn't look like any distribution that I know of.


2. Is the exponential prior conjugate to the exponential likelihood?

No. The posterior is not an exponential. But I'm not sure if I computed the posterior correctly.


I assume that I'm doing something wrong, because I'm later asked about the posterior mean, which is difficult to do by integration (integration by parts and extracting a term that depends upon the Gamma function).

$\endgroup$

1 Answer 1

3
$\begingroup$

Hints:

  1. Your posterior density is only proportionate to that expression, not equal

  2. Both your prior density and posterior density (when sorted out) are Gamma distributions

  3. The conjugate family for the rate of an exponential distribution are Gamma distributions

$\endgroup$
2
  • $\begingroup$ To clarify: "The conjugate family for the rate of an exponential distribution" means that the conjugate for p(theta | D, lambda) should be a gamma function? $\endgroup$ May 7, 2020 at 17:21
  • 1
    $\begingroup$ I might not use your notation because I think of the prior as a Gamma distribution rather than an exponential distribution (in your example with shape parameter $1$ and rate parameter $\lambda$) and I might prefer $\pi$ for the density of $\theta$ rather than $p$, but essentially yes. $\endgroup$
    – Henry
    May 7, 2020 at 17:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.