3
$\begingroup$

I have $X_i \overset{iid}{\sim} N(\theta, \theta^2)$, $\theta >0$, $i=1, \cdots, n$. I would like to find the MLE for $\theta$.

My attempt

I have a log likelihood $l(\theta) = -\frac{n}{2} \log(2 \pi) - \frac{\sum x_i^2}{2 \theta^2} + \frac{\sum x_i}{\theta} - \frac{n}{2}$.

$$ \begin{aligned} \frac{\partial}{\partial \theta} l(\theta) &= \sum x_i^2/\theta^3 - \sum x_i/\theta^2 \\ \frac{\partial^2}{\partial \theta^2} l(\theta) &= -3\sum x_i^2/\theta^4 + 2 \sum x_i/\theta^3 \\ &= \frac{-3 \sum x_i^2 + 2 \sum x_i \theta} {\theta^4} \end{aligned} $$

From $\sum x_i^2/n > \left( \sum x_i \right/n)^2$, I have when $0 < \theta < \frac{3}{2n} \sum x_i$, $\frac{\partial^2}{\partial \theta^2} l(\theta) < 0$

So by setting $\frac{\partial}{\partial \theta} l(\theta) = 0$, I have $\hat{\theta}^{MLE} = \sum x_i^2 / {\sum x_i}$ when $\sum x_i >0 $,

But I can't proceed with the cases $\sum x_i < 0$ or $\frac{3}{2n} \sum x_i < \theta$.

$\endgroup$
2

2 Answers 2

5
$\begingroup$

That calculation doesn't look quite right.

Given the sample $(x_1,\ldots,x_n)$, likelihood function of $\theta$ is

\begin{align} L(\theta\mid x_1,\ldots,x_n)&=\frac{1}{(\theta\sqrt{2\pi})^n}\exp\left[-\frac{1}{2\theta^2}\sum_{i=1}^n(x_i-\theta)^2\right]\quad,\theta>0 \end{align}

So the log-likelihood is of the form

$$\ell(\theta)=\text{constant}-n\ln\theta-\frac{1}{2\theta^2}\sum_{i=1}^n(x_i-\theta)^2$$

, so that

$$\frac{\partial\ell}{\partial\theta}=\frac{-n}{\theta}-\frac{n\bar x}{\theta^2}+\frac{\sum_{i=1}^nx_i^2}{\theta^3}$$

Setting $\dfrac{\partial\ell}{\partial\theta}=0$ and ignoring negative value of $\theta$ yields

$$\hat\theta=-\frac{\bar X}{2}+\sqrt{\frac{\sum_{i=1}^n X_i^2}{n}+\frac{\bar X^2}{4}}$$

Verify that this is indeed the MLE by checking $$\frac{\partial^2\ell}{\partial\theta^2}\mid_{\theta=\hat\theta}<0$$

Finally we should arrive at

$$\hat\theta_{\text{MLE}}=-\frac{\bar X}{2}+\sqrt{\frac{\sum_{i=1}^n X_i^2}{n}+\frac{\bar X^2}{4}}\quad,\text{ if }\bar X>0$$

$\endgroup$
4
  • $\begingroup$ Since I'm not that good at calculus, I wonder if it's sufficient to check the second derivative is negative for the zero of the first derivative. Can we say the method yields a unique, global solution? $\endgroup$
    – moreblue
    Sep 30, 2018 at 7:40
  • 2
    $\begingroup$ @moreblue Yes, $\frac{\partial^2\ell}{\partial\theta^2}\mid_{\theta=\hat\theta}<0$ is a sufficient condition for existence of maxima at $\theta=\hat\theta$ by the second derivative test. I think we can be sure that this maxima is in fact the global maxima. $\endgroup$ Sep 30, 2018 at 7:56
  • 1
    $\begingroup$ To be sure that the maxima is global, I think to check $\lim_{\theta \to 0+}l(\theta) = -\infty$, $\lim_{\theta \to +\infty}l(\theta) = -\infty$ might be sufficient. Am I right? $\endgroup$
    – moreblue
    Sep 30, 2018 at 8:28
  • 1
    $\begingroup$ @moreblue Drawing the log-likelihood might give a better indication to that end, although I am not sure if it is easily done. We also have the first partial derivative test at hand, so by checking the signs of $\frac{\partial\ell}{\partial\theta}$, we can conclude regarding point of maxima. $\endgroup$ Sep 30, 2018 at 8:40
1
$\begingroup$

Since $\theta >0$ it suffices to show the numerator is always negative. Note that you only need to show it's negative at the MLE, not all values of $\theta$.

$$-3\sum x_i^2 + 2 \sum x_i \theta <0$$

Substitute in the MLE to obtain

$$-3 \sum_i x_i^2 + 2\sum_i x_i \frac{\sum_j x_j^2}{\sum_j x_j}$$

Cancel off the two sums on the right hand term

$$=-3\sum_i x_i^2 + 2 \sum_j x_j^2$$

$$=-1 \sum_i x_i^2 <0$$

Which is negative since every summand $x_i^2 >0$

Hence since the second derivative is negative at $\theta^{MLE}$, it's a local maximum of the likelihood function.


As pointed out by @StubbonAtom your derived MLE is incorrect. For example, examine the following R code with $\theta = 2$, the estimator you derived gives $\hat{\theta} \approx 4$.

x<- rnorm(10000,2,2)
sum(x^2)/sum(x)
[1] 4.009042
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.