I have been trying to solve the following problem:

Suppose $X_1,...,X_n$ are iid exponential random variables, with density $f(x;\theta) =\theta e^{-\theta x}$ ,and let us suppose that we have a prior on $\theta$ with density $\theta^{\alpha -1}e^{-\beta \theta}$

Now, I found the posterior to be

$$\theta^{n+\alpha-1}e^{-n \theta \bar{x} - \beta \theta}$$

I now want to find the posterior mean, which I have read is given by:

$$\int \theta \theta^{n+\alpha-1}e^{-n \theta \bar{x} - \beta \theta} d\theta= \int\theta^{n+\alpha}e^{-n \theta \bar{x} - \beta \theta} d\theta$$

I have tried to solve this integral, but I don't end up with anything meaningful. I have tried to read some examples online, and can see that the normalising constant plays a role, but I don't see how it all links together.

  • 2
    $\begingroup$ What class of distributions does your prior on $\theta$ belong to? Can you (perhaps after rewriting) recognize the density of your posterior to belong to a specific class of distributions? $\endgroup$ – Stephan Kolassa Oct 18 '20 at 10:06
  • $\begingroup$ @StephanKolassa I can see they both look like Gamma distributions. For the posterior, I'm unsure how to find the exact Gamma distribution. $\endgroup$ – Bill Oct 18 '20 at 10:17
  • 2
    $\begingroup$ Very good. You have found that your posterior (and your prior) is a distribution that is proportional to a specific gamma distribution. How many possibilities are there for a distribution that is proportional to a given distribution? (This is where the constant comes in. How?) $\endgroup$ – Stephan Kolassa Oct 18 '20 at 10:26
  • 1
    $\begingroup$ Hint: The posterior mean is the expectation of the posterior distribution. A mere look at the Gamma Wikipedia page should enlighten you... $\endgroup$ – Xi'an Oct 19 '20 at 8:32
  • 1
    $\begingroup$ The posterior density IS A PROBABILITY DENSITY, hence it contains the normalising constant, otherwise it would not integrate to ONE! $\endgroup$ – Xi'an Oct 20 '20 at 7:53

Suppose that $X_{1},\ldots,X_{n}$ are iid exponential random variables, with density function $f(x;\theta)=\theta e^{-\theta x}$. Then the likelihood function will be \begin{equation*} \text{L}(\theta|x)=\prod_{i=1}^{n}f(x_{i};\theta)=\prod_{i=1}^{n}\theta e^{-\theta x_{i}}=\theta^{n} e^{-\theta n\bar{x}} \end{equation*} where $n\bar{x}=\sum_{i=1}^{n}x_{i}.$

Now, suppose that we want to use a gamma prior for a quantity $\theta$. So $\theta \sim \text{Ga}(\alpha,\beta).$ Then the prior density function for $\theta$ is \begin{equation*} \pi(\theta)=\frac{\beta^{\alpha}}{\Gamma(\alpha)}\theta^{\alpha-1}e^{-\theta \beta} \end{equation*} therefore, the posterior density function for $\theta$, $\pi(\theta|x)$ is proportional to the prior density times the likelihood function. We can write $\pi(\theta|x)$ as follows \begin{equation*} \pi(\theta|x)=\frac{\pi(\theta)\text{L}(\theta|x)}{\int_{\Theta}\pi(\theta)\text{L}(\theta|x)d\theta}. \end{equation*} In order to compute the posterior mean for $\theta$, say $\text{E}(\theta|x)$. We have \begin{equation*} \text{E}(\theta|x)=\frac{\int \theta \pi(\theta)\text{L}(\theta|x)d\theta}{\int \pi(\theta)\text{L}(\theta|x)d\theta}. \end{equation*} We can calculate all these integrals analytically when the prior distribution is conjugate (if the posterior distribution and the prior belong to the same family of distributions, then the prior is called a conjugate prior) to the likelihood.

Let us first deal with the denominator of the posterior mean $\text{E}(\theta|x)$ [normalising constant]. So, \begin{align*} \frac{\beta^{\alpha}}{\Gamma(\alpha)}\int_{0}^{\infty}\theta^{\alpha-1}e^{-\theta \beta} \theta^{n}e^{-n\bar{x}\theta}d\theta&=\frac{\beta^{\alpha}}{\Gamma(\alpha)}\int_{0}^{\infty}\theta^{(\alpha+n)-1}e^{-\theta (\beta+n\bar{x})}d\theta\\ &=\frac{\beta^{\alpha}}{\Gamma(\alpha)}.\frac{\Gamma(\alpha+n)}{(\beta+n\bar{x})^{\alpha+n}}. \end{align*} Secondly, the integral in the numerator will be \begin{align*} \frac{\beta^{\alpha}}{\Gamma(\alpha)}\int_{0}^{\infty}\theta \theta^{\alpha-1}e^{-\theta \beta} \theta^{n}e^{-n\bar{x}\theta}d\theta&=\frac{\beta^{\alpha}}{\Gamma(\alpha)}\int_{0}^{\infty}\theta^{(\alpha+n+1)-1}e^{-\theta (\beta+n\bar{x})}d\theta\\ &=\frac{\beta^{\alpha}}{\Gamma(\alpha)}.\frac{\Gamma(\alpha+n+1)}{(\beta+n\bar{x})^{\alpha+n+1}}. \end{align*} As a result, the posterior mean, $\text{E}(\theta|x)$ is \begin{equation*} \text{E}(\theta|x)=\frac{\frac{\beta^{\alpha}}{\Gamma(\alpha)}.\frac{\Gamma(\alpha+n+1)}{(\beta+n\bar{x})^{\alpha+n+1}}}{\frac{\beta^{\alpha}}{\Gamma(\alpha)}.\frac{\Gamma(\alpha+n)}{(\beta+n\bar{x})^{\alpha+n}}}=\frac{\Gamma(\alpha+n+1)}{(\beta+n\bar{x})^{\alpha+n+1}}.\frac{(\beta+n\bar{x})^{\alpha+n}}{\Gamma(\alpha+n)}=\frac{\alpha+n}{\beta+n\bar{x}}. \end{equation*}

  • $\begingroup$ I hope this would be useful. All the best Harry. $\endgroup$ – Wael Al-Taie Oct 25 '20 at 23:46
  • 1
    $\begingroup$ You appear to be using two different accounts. You should probably merge the other account with the present one. $\endgroup$ – chl Oct 26 '20 at 9:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.