I want to calculate the Bayesian posterior distribution of an exponential distribution where $\lambda$ is distributed according to gamma distribution. I know that I need to calculate $\frac{P(x|\lambda)P(\lambda)}{P(x)}$. I know $P(\lambda)$ because I know the gamma distribution function. I also know that $P(x|\lambda)$ = $\lambda \exp(-\lambda x)$. But I dont know what $P(x)$ is or what it means.

Am I on the right track? What does $P(x)$ mean in this context?


Typically, Bayesians talk about a "posterior" distribution in light of an update. There are two possible things you could be trying to do here, which are fairly closely related so I'll step through them both.

First is a hierarchical modeling problem: you know the distribution of $x$ given a parameter $\lambda$ ($P(x|\lambda) \sim$ exponential), and you have a distribution on that parameter ($P(\lambda) \sim$ gamma), but if you want the distribution on $x$ by itself ($P(x) \sim$ ?), you need to combine those two pieces of information.

This is done by integration; any value of $x$ could be achieved by any value of $\lambda$, and so $P(x)=\int_0^\infty P(x|\lambda)P(\lambda)d\lambda$. (Since the gamma and exponential distributions are closely related, this actually flows pretty smoothly.)

Second is the problem of inferring what the uncertain value $\lambda$ is, given we observe a particular value $x$ from the resulting exponential distribution. Here the role of $P(x)$ becomes clear--we are conditioning on our observation $x$, and dividing by $P(x)$ normalizes the probability distribution.

As pointed out in @ebb-earl-co's answer, you don't actually need to calculate $P(x)$ in this situation, if you're willing to integrate the distribution $P(x|\lambda)P(\lambda)$ in order to determine what normalizing constant you need to apply to make it a well-formed probability. The 'magic' of Bayes is that this normalizing constant is necessarily the probability of the observation that you're updating on.


$P(x)$ is the marginal density of $X$, which is called the marginal because you get it by marginalizing (i.e. integrating) out $\lambda$ from the joint density of $X$ and $\lambda$. I am going to rename $P(x)$ as $h(x)$ so that it doesn't get confused with the posterior. Then, I'll write the data model as $f(x|\lambda)$ and the prior as $\pi(\lambda)$. Therefore, your expression is $$p(\lambda|x) = \frac{f(x|\lambda)\pi(\lambda)}{h(x)}$$ Now, the beauty of Bayes' Theorem is that you don't need to know $h(x)$ in order to calculate $P(\lambda|x)$! So, it suffices to know that the posterior is proportional to the product of the data model (a.k.a. likelihood) and the prior on $\lambda$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.