2
$\begingroup$

I have some troubles with the following.

I'm trying to find a MLE for $\alpha$, when $X_i \text{~} Exp(\theta)$,$\mu=1/\theta$ and $\log(\mu)=\alpha$. Also $n=30$.

i.e.

$\mu=e^{\alpha}=1/\theta$

$\implies \log(\mu)=\log(1)-\log(\theta)=\alpha$
$\implies \log(\theta)=-\alpha$
$\implies \theta=e^{- \alpha}$

where $\mu$ is the mean.

What I've then got is:

$L(\alpha)=\prod_{i=1}^n [e^{-\alpha} e^{-e^{-\alpha} x_i }]$ $=[e^{-n\alpha} e^{-e^{-\alpha} \sum{x_i} }]$

$l(\alpha)=\log([e^{-n\alpha} e^{-e^{-\alpha} \sum{x_i} }])$ $=-n \alpha -e^{-\alpha} \sum{x_i}$

$\frac{\partial l(\alpha)}{\partial \alpha}=-n-\sum{x_i}\frac{\partial e^{u} }{\partial u}\frac{- \alpha}{\partial \alpha}$ $=-n+e^{- \alpha}\sum{x_i}$

$V(\alpha)=-n+e^{- \alpha}\sum{x_i}$
and then I've been given the Jacobian:
$J(\alpha)=n$

Are my equations correct?

Then I want to use Newton's method:

$$\alpha^{(k+1)}=\alpha^{(k)}-\frac{V(\alpha^{(k)})}{J(\alpha^{(k)})}$$

But using

$V(\alpha)=-n+e^{- \alpha}\sum{x_i}$ does not converge, whereas using

$V(\alpha)=-n+e^{\alpha}\sum{x_i}$

gives the opposite sign:

enter image description here

since by gammer's answer the answer should be $\hat{\alpha}=-0.2514...$

$\endgroup$
0
1
$\begingroup$

If $X_1, ..., X_n \sim {\rm exponential}(\theta)$ then the MLE for $\theta$ is $1/\overline{X}$, where $\overline{X} = \frac{1}{n} \sum_{i=1}^{n} X_i$.

By the functional invariance property of MLEs the MLE of $\alpha = \log(1/\theta)$ is therefore $\hat{\alpha} = \log(\overline{X})$.

$\endgroup$
6
  • $\begingroup$ But can I not approximate the MLE like I'm doing it? Rather, would I need to perform Newton on $\theta$ and then transform that to $\alpha$? Rather than computing in $\alpha$ directly? $\endgroup$ – mavavilj Apr 2 '17 at 3:05
  • $\begingroup$ I was giving the analytic solution to your question. Of course you can compute it numerically if you re-parameterize in terms of $\alpha$. Your equation for the likelihood looks right. I suspect you've made an error in calculating the derivative. You'll know you did it right if the answer you get equals $\log(\overline{X})$... $\endgroup$ – gammer Apr 2 '17 at 3:09
  • $\begingroup$ There's your problem. You're presumably trying to find roots of the score function. In that case, the steps in Newton-Raphson are supposed to be $f'(\alpha_k)/f''(\alpha_k)$. In place of $f'(\alpha_k)$ in your steps you have $f(\alpha_k)$, and where $f''(\alpha_k)$ should be you have $n$, neither of which are right. $\endgroup$ – gammer Apr 2 '17 at 3:13
  • $\begingroup$ I'm using $\hat{\theta}^{(k+1)}=\hat{\theta}^{(k)}-\mathbb{H}^{-1}(\hat{\theta}^{(k)})V(\hat{\theta}^{(k)})$ version of Newton. $\endgroup$ – mavavilj Apr 2 '17 at 3:15
  • $\begingroup$ Just ditch the numerical approximation and compute $\log(\overline{X})$. And it's up to you whether or not you want to delete the question. I'll leave the answer because it's right. $\endgroup$ – gammer Apr 2 '17 at 3:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.