Let $X_{1},X_{2},X_{3},...,X_{n}$ be a random sample from a distribution with pdf $$f(x;\alpha,\theta)=\frac{e^{-x/\theta}}{\theta^{\alpha}\Gamma(\alpha)}x^{\alpha-1}I_{(0,\infty)}(x ),\alpha,\theta>0$$

Find the maximum likelihood estimator of $\alpha$ and $\theta$. Let $\Psi(\alpha)=\frac{d\Gamma(\alpha)}{d\alpha}$

My attempt, \begin{eqnarray*} \mathcal{L}(\alpha,\theta)&=&\prod_{i=1}^{n}f(x_i)\\ &=&\prod_{i=1}^{n}\frac{e^{-x_i/\theta}}{\theta^{\alpha}\Gamma(\alpha)}x_i^{\alpha-1}\\ &=&\frac{1}{\Gamma^{n}(\alpha)\cdot \theta^{n \alpha}}(\prod_{i=1}^{n}x_i)^{\alpha-1}\exp(-\sum_{i=1}^{n}\frac{x_i}{\theta}) \end{eqnarray*} \begin{eqnarray*} \ell(\alpha,\theta)&=&-n\log(\Gamma(\alpha))-n\alpha\log(\theta)+(\alpha-1)\sum_{i=1}^{n}\log(x_i)-\frac{1}{\theta}\sum_{i=1}^{n}x_i\\ \frac{\delta \ell(\alpha,\theta)}{\delta \theta}&=&-\frac{n\alpha}{\theta}+\frac{1}{\theta^2}\sum_{i=1}^{n}x_i=0\\ \frac{1}{\theta^2}\sum_{i=1}^{n}x_i&=&\frac{n\alpha}{\theta}\\ \hat{\theta}&=&\frac{\sum_{i=1}^{n}x_i}{n\alpha}\\ &=&\frac{1}{\alpha}\bar{x}\\ \end{eqnarray*} \begin{eqnarray*} \frac{d \ell(\alpha,\hat{\theta})}{d\alpha}&=&\frac{-n \cdot \Gamma'(\alpha)}{\Gamma(\alpha)}-n\log(\frac{1}{\alpha}\bar{x})+\sum_{i=1}^{n}\log(x_i)=0\\ &=&\frac{-n \cdot \Gamma'(\alpha)}{\Gamma(\alpha)}+n\log(\alpha)-n\log(\bar{x})+\sum_{i=1}^{n}\log(x_i)=0\\ \log(\alpha)-\frac{\Gamma'(\alpha)}{\Gamma(\alpha)}&=&\log(\bar{x})-\frac{\sum_{i=1}^{n}\log(x_i)}{n} \end{eqnarray*}

I could't proceed anymore for finding the $\alpha$. Second, I do not know how to use $\Psi(\alpha)=\frac{d\Gamma(\alpha)}{d\alpha}$ as given in the question. Hope someone can explain it to me.

Thanks in advance.

  • 3
    $\begingroup$ One would have to resort to numerical methods for finding the MLE of $\alpha$. $\endgroup$ – StubbornAtom Jun 19 '18 at 16:26
  • $\begingroup$ You write $\Psi(\alpha)$ as $\Gamma'(\alpha)$ $\endgroup$ – Taylor Jun 19 '18 at 16:29
  • 1
    $\begingroup$ could you have been told to use the digamma function $\psi(x) = \frac{\Gamma'(x)}{\Gamma(x)}$ instead of your $\Psi$? That'd make more sense to me $\endgroup$ – jld Jun 19 '18 at 16:53
  • $\begingroup$ @Chaconne I think the question has typo. It Should be psi instead of Psi. $\endgroup$ – Mathxx Jun 19 '18 at 17:00
  • $\begingroup$ @Taylor I thought $\psi(\alpha)=\frac{\Gamma'(\alpha)}{\Gamma(\alpha)}$? $\endgroup$ – Mathxx Jun 21 '18 at 8:51

Let $\psi(\alpha) = \frac{\Gamma'(\alpha)}{\Gamma(\alpha)}$ so $\psi$ is the digamma function (I'm using $\psi$ rather than your $\Psi$).

By the AM-GM inequality $$ \bar x \geq \left(\prod_i x_i\right)^{1/n} $$ so $$ \log \bar x - \overline{\log x} \geq 0 $$ (where $\log \bar x$ and $\log x_i$ are defined almost surely). Furthermore, equality only holds for $x_1=\dots=x_n$ which is a probability $0$ event, so $\log \bar x - \overline{\log x} > 0$ almost surely.

For simplicity, I'll take $y = \log \bar x - \overline{\log x}$.

Consider $f(\alpha) = \log(\alpha) - \psi(\alpha)$ on $(0,\infty)$. This is continuous and $$ \lim_{\alpha\to 0} f(\alpha) = \infty $$ $$ \lim_{\alpha\to\infty} f(\alpha) = 0 $$ so by the intermediate value theorem $f$ hits every real number in $(0,\infty)$. In particular, this means that $$ f^{-1}\left(\left\{y\right\}\right) \neq \emptyset $$ i.e. there is at least one point in $(0,\infty)$ mapped to $y$, since $y > 0$.

Furthermore, $f$ turns out to be injective on $(0,\infty)$ as $f' < 0$ so there is actually a unique $\hat \alpha$ with $f(\hat\alpha) = y$.

Actually finding this $\hat \alpha$ will require numerical methods though, as @StubbornAtom says.

  • 4
    $\begingroup$ +1 Nice analysis. Given the asymptotics of $\Gamma$ and $\psi,$ I would warmly recommend finding the solution of $\log(x) - \psi(x) = C$ by numerically finding the root of $$\frac{1}{\log(x) - \psi(x)} - \frac{1}{C},$$ because this is going to be extremely close to linear (with slope $2$ at large values). $\endgroup$ – whuber Jun 19 '18 at 17:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.