# MLE of $f(x;\alpha,\theta)=\frac{e^{-x/\theta}}{\theta^{\alpha}\Gamma(\alpha)}x^{\alpha-1}$

Let $X_{1},X_{2},X_{3},...,X_{n}$ be a random sample from a distribution with pdf $$f(x;\alpha,\theta)=\frac{e^{-x/\theta}}{\theta^{\alpha}\Gamma(\alpha)}x^{\alpha-1}I_{(0,\infty)}(x ),\alpha,\theta>0$$

Find the maximum likelihood estimator of $\alpha$ and $\theta$. Let $\Psi(\alpha)=\frac{d\Gamma(\alpha)}{d\alpha}$

My attempt, \begin{eqnarray*} \mathcal{L}(\alpha,\theta)&=&\prod_{i=1}^{n}f(x_i)\\ &=&\prod_{i=1}^{n}\frac{e^{-x_i/\theta}}{\theta^{\alpha}\Gamma(\alpha)}x_i^{\alpha-1}\\ &=&\frac{1}{\Gamma^{n}(\alpha)\cdot \theta^{n \alpha}}(\prod_{i=1}^{n}x_i)^{\alpha-1}\exp(-\sum_{i=1}^{n}\frac{x_i}{\theta}) \end{eqnarray*} \begin{eqnarray*} \ell(\alpha,\theta)&=&-n\log(\Gamma(\alpha))-n\alpha\log(\theta)+(\alpha-1)\sum_{i=1}^{n}\log(x_i)-\frac{1}{\theta}\sum_{i=1}^{n}x_i\\ \frac{\delta \ell(\alpha,\theta)}{\delta \theta}&=&-\frac{n\alpha}{\theta}+\frac{1}{\theta^2}\sum_{i=1}^{n}x_i=0\\ \frac{1}{\theta^2}\sum_{i=1}^{n}x_i&=&\frac{n\alpha}{\theta}\\ \hat{\theta}&=&\frac{\sum_{i=1}^{n}x_i}{n\alpha}\\ &=&\frac{1}{\alpha}\bar{x}\\ \end{eqnarray*} \begin{eqnarray*} \frac{d \ell(\alpha,\hat{\theta})}{d\alpha}&=&\frac{-n \cdot \Gamma'(\alpha)}{\Gamma(\alpha)}-n\log(\frac{1}{\alpha}\bar{x})+\sum_{i=1}^{n}\log(x_i)=0\\ &=&\frac{-n \cdot \Gamma'(\alpha)}{\Gamma(\alpha)}+n\log(\alpha)-n\log(\bar{x})+\sum_{i=1}^{n}\log(x_i)=0\\ \log(\alpha)-\frac{\Gamma'(\alpha)}{\Gamma(\alpha)}&=&\log(\bar{x})-\frac{\sum_{i=1}^{n}\log(x_i)}{n} \end{eqnarray*}

I could't proceed anymore for finding the $\alpha$. Second, I do not know how to use $\Psi(\alpha)=\frac{d\Gamma(\alpha)}{d\alpha}$ as given in the question. Hope someone can explain it to me.

• One would have to resort to numerical methods for finding the MLE of $\alpha$. – StubbornAtom Jun 19 '18 at 16:26
• You write $\Psi(\alpha)$ as $\Gamma'(\alpha)$ – Taylor Jun 19 '18 at 16:29
• could you have been told to use the digamma function $\psi(x) = \frac{\Gamma'(x)}{\Gamma(x)}$ instead of your $\Psi$? That'd make more sense to me – jld Jun 19 '18 at 16:53
• @Chaconne I think the question has typo. It Should be psi instead of Psi. – Mathxx Jun 19 '18 at 17:00
• @Taylor I thought $\psi(\alpha)=\frac{\Gamma'(\alpha)}{\Gamma(\alpha)}$? – Mathxx Jun 21 '18 at 8:51

Let $\psi(\alpha) = \frac{\Gamma'(\alpha)}{\Gamma(\alpha)}$ so $\psi$ is the digamma function (I'm using $\psi$ rather than your $\Psi$).

By the AM-GM inequality $$\bar x \geq \left(\prod_i x_i\right)^{1/n}$$ so $$\log \bar x - \overline{\log x} \geq 0$$ (where $\log \bar x$ and $\log x_i$ are defined almost surely). Furthermore, equality only holds for $x_1=\dots=x_n$ which is a probability $0$ event, so $\log \bar x - \overline{\log x} > 0$ almost surely.

For simplicity, I'll take $y = \log \bar x - \overline{\log x}$.

Consider $f(\alpha) = \log(\alpha) - \psi(\alpha)$ on $(0,\infty)$. This is continuous and $$\lim_{\alpha\to 0} f(\alpha) = \infty$$ $$\lim_{\alpha\to\infty} f(\alpha) = 0$$ so by the intermediate value theorem $f$ hits every real number in $(0,\infty)$. In particular, this means that $$f^{-1}\left(\left\{y\right\}\right) \neq \emptyset$$ i.e. there is at least one point in $(0,\infty)$ mapped to $y$, since $y > 0$.

Furthermore, $f$ turns out to be injective on $(0,\infty)$ as $f' < 0$ so there is actually a unique $\hat \alpha$ with $f(\hat\alpha) = y$.

Actually finding this $\hat \alpha$ will require numerical methods though, as @StubbornAtom says.

• +1 Nice analysis. Given the asymptotics of $\Gamma$ and $\psi,$ I would warmly recommend finding the solution of $\log(x) - \psi(x) = C$ by numerically finding the root of $$\frac{1}{\log(x) - \psi(x)} - \frac{1}{C},$$ because this is going to be extremely close to linear (with slope $2$ at large values). – whuber Jun 19 '18 at 17:36