4
$\begingroup$

I have been trying to solve the following problem:

Suppose $X_1,...,X_n$ are iid exponential random variables, with density $f(x;\theta) =\theta e^{-\theta x}$ ,and let us suppose that we have a prior on $\theta$ with density $\theta^{\alpha -1}e^{-\beta \theta}$

Now, I found the posterior to be

$$\theta^{n+\alpha-1}e^{-n \theta \bar{x} - \beta \theta}$$

I now want to find the posterior mean, which I have read is given by:

$$\int \theta \theta^{n+\alpha-1}e^{-n \theta \bar{x} - \beta \theta} d\theta= \int\theta^{n+\alpha}e^{-n \theta \bar{x} - \beta \theta} d\theta$$

I have tried to solve this integral, but I don't end up with anything meaningful. I have tried to read some examples online, and can see that the normalising constant plays a role, but I don't see how it all links together.

$\endgroup$
6
  • 2
    $\begingroup$ What class of distributions does your prior on $\theta$ belong to? Can you (perhaps after rewriting) recognize the density of your posterior to belong to a specific class of distributions? $\endgroup$ Oct 18, 2020 at 10:06
  • $\begingroup$ @StephanKolassa I can see they both look like Gamma distributions. For the posterior, I'm unsure how to find the exact Gamma distribution. $\endgroup$
    – Sparsity
    Oct 18, 2020 at 10:17
  • 2
    $\begingroup$ Very good. You have found that your posterior (and your prior) is a distribution that is proportional to a specific gamma distribution. How many possibilities are there for a distribution that is proportional to a given distribution? (This is where the constant comes in. How?) $\endgroup$ Oct 18, 2020 at 10:26
  • 1
    $\begingroup$ Hint: The posterior mean is the expectation of the posterior distribution. A mere look at the Gamma Wikipedia page should enlighten you... $\endgroup$
    – Xi'an
    Oct 19, 2020 at 8:32
  • 1
    $\begingroup$ The posterior density IS A PROBABILITY DENSITY, hence it contains the normalising constant, otherwise it would not integrate to ONE! $\endgroup$
    – Xi'an
    Oct 20, 2020 at 7:53

1 Answer 1

2
$\begingroup$

Suppose that $X_{1},\ldots,X_{n}$ are iid exponential random variables, with density function $f(x;\theta)=\theta e^{-\theta x}$. Then the likelihood function will be \begin{equation*} \text{L}(\theta|x)=\prod_{i=1}^{n}f(x_{i};\theta)=\prod_{i=1}^{n}\theta e^{-\theta x_{i}}=\theta^{n} e^{-\theta n\bar{x}} \end{equation*} where $n\bar{x}=\sum_{i=1}^{n}x_{i}.$

Now, suppose that we want to use a gamma prior for a quantity $\theta$. So $\theta \sim \text{Ga}(\alpha,\beta).$ Then the prior density function for $\theta$ is \begin{equation*} \pi(\theta)=\frac{\beta^{\alpha}}{\Gamma(\alpha)}\theta^{\alpha-1}e^{-\theta \beta} \end{equation*} therefore, the posterior density function for $\theta$, $\pi(\theta|x)$ is proportional to the prior density times the likelihood function. We can write $\pi(\theta|x)$ as follows \begin{equation*} \pi(\theta|x)=\frac{\pi(\theta)\text{L}(\theta|x)}{\int_{\Theta}\pi(\theta)\text{L}(\theta|x)d\theta}. \end{equation*} In order to compute the posterior mean for $\theta$, say $\text{E}(\theta|x)$. We have \begin{equation*} \text{E}(\theta|x)=\frac{\int \theta \pi(\theta)\text{L}(\theta|x)d\theta}{\int \pi(\theta)\text{L}(\theta|x)d\theta}. \end{equation*} We can calculate all these integrals analytically when the prior distribution is conjugate (if the posterior distribution and the prior belong to the same family of distributions, then the prior is called a conjugate prior) to the likelihood.

Let us first deal with the denominator of the posterior mean $\text{E}(\theta|x)$ [normalising constant]. So, \begin{align*} \frac{\beta^{\alpha}}{\Gamma(\alpha)}\int_{0}^{\infty}\theta^{\alpha-1}e^{-\theta \beta} \theta^{n}e^{-n\bar{x}\theta}d\theta&=\frac{\beta^{\alpha}}{\Gamma(\alpha)}\int_{0}^{\infty}\theta^{(\alpha+n)-1}e^{-\theta (\beta+n\bar{x})}d\theta\\ &=\frac{\beta^{\alpha}}{\Gamma(\alpha)}.\frac{\Gamma(\alpha+n)}{(\beta+n\bar{x})^{\alpha+n}}. \end{align*} Secondly, the integral in the numerator will be \begin{align*} \frac{\beta^{\alpha}}{\Gamma(\alpha)}\int_{0}^{\infty}\theta \theta^{\alpha-1}e^{-\theta \beta} \theta^{n}e^{-n\bar{x}\theta}d\theta&=\frac{\beta^{\alpha}}{\Gamma(\alpha)}\int_{0}^{\infty}\theta^{(\alpha+n+1)-1}e^{-\theta (\beta+n\bar{x})}d\theta\\ &=\frac{\beta^{\alpha}}{\Gamma(\alpha)}.\frac{\Gamma(\alpha+n+1)}{(\beta+n\bar{x})^{\alpha+n+1}}. \end{align*} As a result, the posterior mean, $\text{E}(\theta|x)$ is \begin{equation*} \text{E}(\theta|x)=\frac{\frac{\beta^{\alpha}}{\Gamma(\alpha)}.\frac{\Gamma(\alpha+n+1)}{(\beta+n\bar{x})^{\alpha+n+1}}}{\frac{\beta^{\alpha}}{\Gamma(\alpha)}.\frac{\Gamma(\alpha+n)}{(\beta+n\bar{x})^{\alpha+n}}}=\frac{\Gamma(\alpha+n+1)}{(\beta+n\bar{x})^{\alpha+n+1}}.\frac{(\beta+n\bar{x})^{\alpha+n}}{\Gamma(\alpha+n)}=\frac{\alpha+n}{\beta+n\bar{x}}. \end{equation*}

$\endgroup$
2
  • $\begingroup$ I hope this would be useful. All the best Harry. $\endgroup$ Oct 25, 2020 at 23:46
  • 1
    $\begingroup$ You appear to be using two different accounts. You should probably merge the other account with the present one. $\endgroup$
    – chl
    Oct 26, 2020 at 9:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.