I have exponentially distributed probability of event $E$

$$P(E|a) = a \exp(-aE),$$

where $a$ is the rate parameter of the exponential distribution.

Now the probability distribution for $a$ is a Gamma distribution with parameters $d$ and $b$

$$P(a|d,b) = b^d a^{d-1} \exp(-ba) / \Gamma(d)$$

Knowing the prior distribution over $a$ and the likelihood over $E$, $P(E|a)$, how do I derive the posterior distribution for $a$?

I think (not sure!) that it should of the form $P(a|E,d,b)$. I also think that the result should be a standard probability distribution.

  • 1
    $\begingroup$ Is this for some subject? $\endgroup$
    – Glen_b
    Nov 21 '13 at 23:33
  • $\begingroup$ Since this is standard bookwork. Please read the tag-wiki info for the self-study tag. If it applies (if, for example, this is work for some subject), could you add the tag to your post (I suggest removing the parameterization tag). $\endgroup$
    – Glen_b
    Nov 22 '13 at 0:27

From Bayes' theorem, the posterior is proportional to the prior $\times$ the likelihood.

That is,

$$P(a|E,d,b)\propto P(E|a) P(a|d,b)$$

(which you should be able to show for yourself). When you take the product and collect like terms, you get something that will be a density for $a$, up to a multiplicative constant.

If you simplify that result and can recognize that density (I call it playing 'spot the density'), you can identify the parameters of the density you recognize and so work out the constant required to make it a density, at which point you have the complete posterior density.

In this case it has a nice simple form, that of a density I can say with some certainty you're already aware of.

Since the OP has clearly got the complete answer I will outline an approach to this question for future reference. (It's possible to do it a bit more formally than this.)

\begin{eqnarray} P(a|E,d,b)&\propto& P(E|a) P(a|d,b)\quad \text{ (from Bayes' Theorem)}\\ &\propto& a\exp(−aE) \cdot b^da^{d−1}\exp(−ba)/\Gamma(d)\\ &\propto& a^{d}\exp(−\{b+E\}a)\\ &\propto& a^{d^*-1}\exp(−b^*a) \end{eqnarray}

where $d^* = d+1$ and $b^*=b+E$. This is clearly of the same gamma form that we started with in the prior, but with different shape and rate parameters.

If required, it's a simple matter to write down the constants needed to make it a proper density.

  • $\begingroup$ Thank you for this clarification, this is actually what I did, just needed reassuring that I am on the right track:) $\endgroup$
    – ivanibash
    Nov 21 '13 at 23:39
  • $\begingroup$ I've multiplied the two but having a trouble detecting a distribution. Is it also Gamma? $\endgroup$
    – ivanibash
    Nov 21 '13 at 23:50
  • $\begingroup$ I got ( (b^d) * a^d * e^(-a(b+E)) )/ Г(d) $\endgroup$
    – ivanibash
    Nov 21 '13 at 23:59
  • $\begingroup$ Oh yeah, the exponential is aexp(-ax) and this one is x^dexp(-ax), right? I' m not sure I know a distribution of this form. $\endgroup$
    – ivanibash
    Nov 22 '13 at 0:26
  • $\begingroup$ if I do the same operation on the prior (droping the constants) I get a^(d-1)*exp(-b*a) which very similar to what I got for the posterior. but I'm afraid to make any conclusions now. $\endgroup$
    – ivanibash
    Nov 22 '13 at 0:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.