I have some troubles with the following.

I'm trying to find a MLE for $\alpha$, when $X_i \text{~} Exp(\theta)$,$\mu=1/\theta$ and $\log(\mu)=\alpha$. Also $n=30$.



$\implies \log(\mu)=\log(1)-\log(\theta)=\alpha$
$\implies \log(\theta)=-\alpha$
$\implies \theta=e^{- \alpha}$

where $\mu$ is the mean.

What I've then got is:

$L(\alpha)=\prod_{i=1}^n [e^{-\alpha} e^{-e^{-\alpha} x_i }]$ $=[e^{-n\alpha} e^{-e^{-\alpha} \sum{x_i} }]$

$l(\alpha)=\log([e^{-n\alpha} e^{-e^{-\alpha} \sum{x_i} }])$ $=-n \alpha -e^{-\alpha} \sum{x_i}$

$\frac{\partial l(\alpha)}{\partial \alpha}=-n-\sum{x_i}\frac{\partial e^{u} }{\partial u}\frac{- \alpha}{\partial \alpha}$ $=-n+e^{- \alpha}\sum{x_i}$

$V(\alpha)=-n+e^{- \alpha}\sum{x_i}$
and then I've been given the Jacobian:

Are my equations correct?

Then I want to use Newton's method:


But using

$V(\alpha)=-n+e^{- \alpha}\sum{x_i}$ does not converge, whereas using


gives the opposite sign:

enter image description here

since by gammer's answer the answer should be $\hat{\alpha}=-0.2514...$


1 Answer 1


If $X_1, ..., X_n \sim {\rm exponential}(\theta)$ then the MLE for $\theta$ is $1/\overline{X}$, where $\overline{X} = \frac{1}{n} \sum_{i=1}^{n} X_i$.

By the functional invariance property of MLEs the MLE of $\alpha = \log(1/\theta)$ is therefore $\hat{\alpha} = \log(\overline{X})$.

  • $\begingroup$ But can I not approximate the MLE like I'm doing it? Rather, would I need to perform Newton on $\theta$ and then transform that to $\alpha$? Rather than computing in $\alpha$ directly? $\endgroup$
    – mavavilj
    Commented Apr 2, 2017 at 3:05
  • $\begingroup$ I was giving the analytic solution to your question. Of course you can compute it numerically if you re-parameterize in terms of $\alpha$. Your equation for the likelihood looks right. I suspect you've made an error in calculating the derivative. You'll know you did it right if the answer you get equals $\log(\overline{X})$... $\endgroup$
    – gammer
    Commented Apr 2, 2017 at 3:09
  • $\begingroup$ There's your problem. You're presumably trying to find roots of the score function. In that case, the steps in Newton-Raphson are supposed to be $f'(\alpha_k)/f''(\alpha_k)$. In place of $f'(\alpha_k)$ in your steps you have $f(\alpha_k)$, and where $f''(\alpha_k)$ should be you have $n$, neither of which are right. $\endgroup$
    – gammer
    Commented Apr 2, 2017 at 3:13
  • $\begingroup$ I'm using $\hat{\theta}^{(k+1)}=\hat{\theta}^{(k)}-\mathbb{H}^{-1}(\hat{\theta}^{(k)})V(\hat{\theta}^{(k)})$ version of Newton. $\endgroup$
    – mavavilj
    Commented Apr 2, 2017 at 3:15
  • $\begingroup$ Just ditch the numerical approximation and compute $\log(\overline{X})$. And it's up to you whether or not you want to delete the question. I'll leave the answer because it's right. $\endgroup$
    – gammer
    Commented Apr 2, 2017 at 3:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.