Let $X \sim Gamma(\alpha,1)$ and $Y|X=x \sim Exp(\frac{1}{\theta x}), \alpha >1$ and $\theta >0$ are unknown. Let $\tau=E(Y)$. Suppose that based on the random sample $Y_1,...,Y_n$, we have MLEs, $\hat{\alpha}$ and $\hat{\theta}$. Use these MLEs to develop an asymptotic $1-\alpha$ confidence interval for $\tau$.

my work:

First, I need to find $\tau=E(Y)=E(\frac{1}{\theta x})=\frac{1}{\theta}E(\frac{1}{x})$. We use a transformation of $T=\frac{1}{X}$, where $f_T(t)=\frac{1}{\Gamma(\alpha)t^{\alpha+1}}e^{-1/t},t>0$. However, I am having trouble evaluating $E(T)=\int^\infty_0\frac{1}{\Gamma(\alpha)t^{\alpha}}e^{-1/t}dt$.

Assuming we have $\tau$, we can get the asymptotic $1-\alpha$ CI by using the asymptotic property of MLE. We know that $\hat{\alpha}\sim AN(\alpha,\frac{1}{ni(\alpha)})$ and $\hat{\theta} \sim AN(\theta,\frac{1}{ni(\theta)})$. However, I am failing to see how I can obtain the asymptotic CI for $\tau$.

updated work:

Thanks to Oriol, I get that $\tau=\frac{1}{\theta}\frac{\Gamma(\alpha-1)}{\Gamma(\alpha)}$. I now see that through the invariance property of MLE, $\hat{\tau}=\frac{1}{\hat{\theta}}\frac{\Gamma(\hat{\alpha}-1)}{\Gamma(\hat{\alpha})}$.

We can get our asymptotic $1-\alpha$ confidence interval for $\tau$ with

$\hat{\tau} \pm z_{\alpha/2}\frac{1}{\sqrt{ni(\hat{\tau})}} \implies \hat{\tau} \pm z_{\alpha/2}\sqrt{\hat{Var}(\tau(\hat{\theta},\hat{\alpha})|\theta,\alpha)}$.

To be quite honest, I do not see how to derive either $\sqrt{ni(\hat{\tau})}$ or $\sqrt{\hat{Var}(\tau(\hat{\theta},\hat{\alpha})|\theta,\alpha)}=\sqrt{\hat{V}}$. Up to this point, since regularity conditions hold, I have been using $i(\tau)$ to denote the Fisher information for a single observation and would prefer to see a solution using this form of Fisher information. Regarding the variance term, I know that

$\hat{V} \approx \frac{(\tau'(\hat{\theta},\hat{\alpha}))^2}{-\frac{\partial^2}{\partial \theta \partial \alpha}logL(\hat{\theta},\hat{\alpha}|X)}$,

but I do not know how to derive this term.

  • 1
    $\begingroup$ What are the pdfs of $Exp(a)$ and $Gamma(a,1)$? $\endgroup$ – StubbornAtom Apr 18 '20 at 6:04
  • $\begingroup$ @StubbornAtom $f(y)=\frac{1}{1/(\theta x)}exp(-\frac{y}{1/(\theta x)}), y>0$ and $f(x|\alpha)=\frac{1}{\Gamma(\alpha)}x^{\alpha-1}e^{-x},x>0$, using the scale parametrization for the Gamma distribution $\endgroup$ – Ron Snow Apr 18 '20 at 21:33

I see you are computing $\tau$ using the law of total expectations. Using linearity of expectations and $Y\sim Exp\left(\theta x\right)$ (I see in your comment that this is the pdf of $Y | X=x$. Usually the notation is different) we get: $$\tau = \mathbb{E}_Y[Y]=\mathbb{E}_Y[\mathbb{E}_X[Y|X]]=\mathbb{E}_X[\mathbb{E}_Y[Y|X]]=\mathbb{E}_X\left[\frac{1}{\theta X}\right]$$

but your mistake is that as you see the expectation is taken with respect to $X$ (not its inverse). So you can compute it as:

\begin{align}\tau&=\mathbb{E}_X\left[\frac{1}{\theta X}\right]=\frac{1}{\theta}\int_0^\infty \frac{1}{x} \frac{1}{\Gamma(\alpha)} x^{\alpha - 1} e^{-x}dx \\&= \frac{1}{\theta}\frac{\Gamma(\alpha-1)}{\Gamma(\alpha)}\int_0^\infty \frac{1}{\Gamma(\alpha - 1)} x^{(\alpha - 1) - 1} e^{-x}dx=\frac{1}{\theta}\frac{\Gamma(\alpha-1)}{\Gamma(\alpha)}\end{align} where the last step follows since we are integrating a pdf. Now you can take $\hat{\tau}=\frac{1}{\hat{\theta}}\frac{\Gamma(\hat{\alpha}-1)}{\Gamma(\hat{\alpha})}$.

You can further simplify the latter expression if $\alpha \in \mathbb{N}$, in which case $\Gamma(\alpha)=(\alpha-1)!$ and hence $\hat{\tau}=\frac{1}{\hat{\theta}(\hat{\alpha} - 1)}$.

You can compute the distribution of $Y$ using the law of total probability:

\begin{align} f_Y(y;\alpha,\theta)&=\int_0^\infty f_{Y|X=x}(y) f_X(x) dx = \int_0^\infty \theta x e^{-\theta x y}\frac{1}{\Gamma(\alpha)} x^{\alpha - 1} e^{-x} dx \\ &=\frac{\Gamma(\alpha + 1)\theta}{\Gamma(\alpha)(1+\theta y)^{\alpha + 1}} \int_0^\infty \frac{(1+\theta y)^{\alpha + 1}}{\Gamma(\alpha + 1)} x^{(\alpha + 1) - 1} e^{-x(1 + \theta y)} dx \\ &=\frac{\Gamma(\alpha + 1)\theta}{\Gamma(\alpha)(1+\theta y)^{\alpha + 1}}\end{align}

where I used the same trick of integrating over a pdf, in this case $Ga(\alpha + 1 , (1+\theta y))$. You can check that the result is non-negative and integrates to 1 so it's actually a pdf (recall $y\in[0,\infty)$).

Now you have to derive the MLEs of $\alpha$ and $\theta$ so you can compute the confidence intervals. To do so, you first need to compute the log-likelihood of your sample, which by independence is

\begin{align} \ell(\alpha,\theta)&=\sum_{i=1}^n \log f(y_i ; \alpha, \theta)\\ &=\sum_{i=1}^n \left[ \log \Gamma(\alpha + 1) + \log \theta - \log \Gamma(\alpha) - (\alpha + 1 )\log(1+\theta y_i) \right] \end{align}

Hope you can continue from here.

  • $\begingroup$ Thank you. I see how you derived $\hat{\tau}$. Unfortunately, I cannot assume $\alpha \in N$. I have added updated work to my post to get closer to finding the asymptotic $1-\alpha$ confidence interval, but I do not know where to go from there. $\endgroup$ – Ron Snow Apr 20 '20 at 16:56
  • $\begingroup$ Okay- got it. I'll find a way to use the Fisher information matrix from here! $\endgroup$ – Ron Snow Apr 20 '20 at 18:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.