11
$\begingroup$

I have been attempting to figure this out for hours, but gamma distribution is somehow beyond me. I have a question where we are given α=5 and β=1 for the prior (Gamma(5, 1)). We are also given λ=4 for our poisson distribution and are asked to calculate the value of the posterior and Bayesian Estimate.

So far I attempted this posterior = P(λ) * Gamma(5, 1) = 24 * .0733 = 1.753, a formula I found online. However, I have absolutely no idea if this is correct. I don't even know where to start for the Bayesian Estimate. Can someone please help point me in the right direction?

$\endgroup$
1
  • $\begingroup$ Welcome to Cross Validated! Try to express the problem more clearly & explain how far you've got. For a start, write down the prior probability density function for the Poisson mean & the probability mass function for the observed data conditional on that mean. $\endgroup$ Nov 14, 2014 at 10:27

2 Answers 2

12
$\begingroup$

You can read these Wikipedia pages on conjugate prior and prior probability first. In short, the posterior probability is the probability of parameters $\theta$ given the data x, or $p(\theta|x)$, and prior probability is a probability about the uncertainty of parameters based on subjective assessment, or $p(\theta)$.

Based on the Bayes' theorem, the relationship between the prior, the posterior, and the likelihood function is $p(\theta|x) = \frac{p(x|\theta)p(\theta)}{\int p(x|\theta^{`})p(\theta^{`})}$. For certain choices of $p(\theta)$, the posterior and the prior has same algebraic form, and the integral in the denominator has closed form; such $p(\theta)$ is called a conjugate prior. The bottom of the conjugate prior page shows that the Gamma distribution is a conjugate prior of the Poisson distribution.


Before computing the posterior $p(\lambda|x)$ with prior $g(\lambda;\alpha,\beta) = \frac{\beta^{\alpha} \lambda^{\alpha-1} e^{-\lambda\beta}}{\Gamma(\alpha)}$ and Poisson pmf $p(x|\lambda)= \frac{e^{-\lambda}\lambda^x }{x!}$, what are the samples from the Poisson distribution? You need to have some samples drawn from the Poission distribution to compute the likelihood with them and then compute the posterior.

Suppose the samples are $x = \{x_1, ..., x_n\}$, then $p(x|\lambda) = \frac{\lambda^{\sum{x_i}}* e^{-n\lambda}}{x_1*...x_n}$. It's too troublesome to type the derivation since the likelihood of Poisson samples is fairly complex. The following derivation considers that there's only one sample x and $p(x|\lambda)= \frac{e^{-\lambda}\lambda^x }{x!}$, and you can derive the posterior for n samples own your own and see if the posterior follows a Gamma distribution with parameters $\alpha + \sum_{i=1}^n x_i ,\ \beta + n\!$ as listed in the first link.


$p(\lambda|x)= (\frac{e^{-\lambda}\lambda^x}{x!} * \frac{\beta^{\alpha} \lambda^{\alpha-1} e^{-\lambda\beta}}{\Gamma(\alpha)})\div (\int_0^\infty (\frac{e^{-\lambda}\lambda^x}{x!} * \frac{\beta^{\alpha} \lambda^{\alpha-1} e^{-\lambda\beta}}{\Gamma(\alpha)}) d\lambda)$. The denominator = $\int_0^\infty ( \frac{\beta^{\alpha+x} \lambda^{\alpha+x-1} e^{-\lambda(\beta+1)}}{\beta^x*x!*\Gamma(\alpha)}) d\lambda = \frac{ \Gamma(\alpha +x)}{\beta^x*x!*\Gamma(\alpha)} \int_0^\infty Gamma(\lambda; \alpha + x, \beta+1) d\lambda = \frac{ \Gamma(\alpha +x)}{\beta^x*x!*\Gamma(\alpha)}$.

After canceling some terms in the numerator and denominator, $p(\lambda|x)= \frac{ \beta^{\alpha+x} \lambda^{\alpha+x-1} e^{-\lambda(\beta+1)}}{\Gamma(\alpha+x)} = Gamma(\lambda; \alpha + x, \beta+1)$. Substitute the values $\alpha, \beta, \lambda$ , and x to obtain the posterior probability.

$\endgroup$
2
  • 1
    $\begingroup$ Thank you so much to taking the time out of your day to thoroughly explain gamma distributions. After doing everything from reading through 2 textbooks to watching a Harvard lecture on the topic, this was the most well worded and helpful thing I've come across $\endgroup$
    – BadKarma
    Nov 17, 2014 at 7:18
  • 1
    $\begingroup$ I solved the same exercise yesterday and got the same answer, but I have a question, if the distribution is Gamma($a+x,\beta+1)$ the pdf should not be $p(\lambda|x)= \frac{ (\beta+1)^{\alpha+x} \lambda^{\alpha+x-1} e^{-\lambda(\beta+1)}}{\Gamma(\alpha+x)} = Gamma(\lambda; \alpha + x, \beta+1)$? $\endgroup$
    – user72621
    Mar 9, 2016 at 15:10
4
$\begingroup$

I think the derivation by @Tom is partially correct and there is a value missing from the final expression. \begin{align} \int_0^{\infty}(\frac{\beta^{\alpha}\lambda^{\alpha+x-1}e^{-\lambda(\beta+1)}}{x!\Gamma(\alpha)})d\lambda & = \frac{\Gamma(\alpha+x)}{x!\Gamma(\alpha)}\frac{\beta^\alpha}{(1+\beta)^{\alpha+x}}\int_0^{\infty}\frac{(\beta+1)^{\alpha+x}}{\Gamma(\alpha+x)}\lambda^{\alpha+x-1}e^{-\lambda(\beta+1)}d\lambda \\ & = \frac{\Gamma(\alpha+x)}{x!\Gamma(\alpha)}\frac{\beta^\alpha}{(1+\beta)^{\alpha+x}}\int_0^{\infty}Gamma(\lambda;\alpha+x,\beta+1)d\lambda \\ & = \frac{\Gamma(\alpha+x)}{x!\Gamma(\alpha)}\frac{\beta^\alpha}{(1+\beta)^{\alpha+x}} \end{align} I guess this would be the final form. Now if you plug this as denominator in $p(x|\lambda)$ then after some cancellation we get $Gamma(\lambda ;\alpha+x,\beta+1)$

$\endgroup$
3
  • $\begingroup$ Do you mean $Gamma$ or $\Gamma$ instead? $\endgroup$ Jul 13, 2016 at 19:44
  • 1
    $\begingroup$ I used $Gamma$ to denote gamma distribution and $\Gamma$ as gamma function. $\endgroup$ Jul 14, 2016 at 15:30
  • $\begingroup$ Sorry for my ignorance! $\endgroup$ Jul 14, 2016 at 16:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.