# MLE of $\theta$ in $N(\theta, \theta^2)$

I have $$X_i \overset{iid}{\sim} N(\theta, \theta^2)$$, $$\theta >0$$, $$i=1, \cdots, n$$. I would like to find the MLE for $$\theta$$.

My attempt

I have a log likelihood $$l(\theta) = -\frac{n}{2} \log(2 \pi) - \frac{\sum x_i^2}{2 \theta^2} + \frac{\sum x_i}{\theta} - \frac{n}{2}$$.

\begin{aligned} \frac{\partial}{\partial \theta} l(\theta) &= \sum x_i^2/\theta^3 - \sum x_i/\theta^2 \\ \frac{\partial^2}{\partial \theta^2} l(\theta) &= -3\sum x_i^2/\theta^4 + 2 \sum x_i/\theta^3 \\ &= \frac{-3 \sum x_i^2 + 2 \sum x_i \theta} {\theta^4} \end{aligned}

From $$\sum x_i^2/n > \left( \sum x_i \right/n)^2$$, I have when $$0 < \theta < \frac{3}{2n} \sum x_i$$, $$\frac{\partial^2}{\partial \theta^2} l(\theta) < 0$$

So by setting $$\frac{\partial}{\partial \theta} l(\theta) = 0$$, I have $$\hat{\theta}^{MLE} = \sum x_i^2 / {\sum x_i}$$ when $$\sum x_i >0$$,

But I can't proceed with the cases $$\sum x_i < 0$$ or $$\frac{3}{2n} \sum x_i < \theta$$.

• Sep 30, 2018 at 7:24
• Is this a homework question? Then you may want to tag it as [self-study] Sep 30, 2018 at 8:01

That calculation doesn't look quite right.

Given the sample $$(x_1,\ldots,x_n)$$, likelihood function of $$\theta$$ is

\begin{align} L(\theta\mid x_1,\ldots,x_n)&=\frac{1}{(\theta\sqrt{2\pi})^n}\exp\left[-\frac{1}{2\theta^2}\sum_{i=1}^n(x_i-\theta)^2\right]\quad,\theta>0 \end{align}

So the log-likelihood is of the form

$$\ell(\theta)=\text{constant}-n\ln\theta-\frac{1}{2\theta^2}\sum_{i=1}^n(x_i-\theta)^2$$

, so that

$$\frac{\partial\ell}{\partial\theta}=\frac{-n}{\theta}-\frac{n\bar x}{\theta^2}+\frac{\sum_{i=1}^nx_i^2}{\theta^3}$$

Setting $$\dfrac{\partial\ell}{\partial\theta}=0$$ and ignoring negative value of $$\theta$$ yields

$$\hat\theta=-\frac{\bar X}{2}+\sqrt{\frac{\sum_{i=1}^n X_i^2}{n}+\frac{\bar X^2}{4}}$$

Verify that this is indeed the MLE by checking $$\frac{\partial^2\ell}{\partial\theta^2}\mid_{\theta=\hat\theta}<0$$

Finally we should arrive at

$$\hat\theta_{\text{MLE}}=-\frac{\bar X}{2}+\sqrt{\frac{\sum_{i=1}^n X_i^2}{n}+\frac{\bar X^2}{4}}\quad,\text{ if }\bar X>0$$

• Since I'm not that good at calculus, I wonder if it's sufficient to check the second derivative is negative for the zero of the first derivative. Can we say the method yields a unique, global solution? Sep 30, 2018 at 7:40
• @moreblue Yes, $\frac{\partial^2\ell}{\partial\theta^2}\mid_{\theta=\hat\theta}<0$ is a sufficient condition for existence of maxima at $\theta=\hat\theta$ by the second derivative test. I think we can be sure that this maxima is in fact the global maxima. Sep 30, 2018 at 7:56
• To be sure that the maxima is global, I think to check $\lim_{\theta \to 0+}l(\theta) = -\infty$, $\lim_{\theta \to +\infty}l(\theta) = -\infty$ might be sufficient. Am I right? Sep 30, 2018 at 8:28
• @moreblue Drawing the log-likelihood might give a better indication to that end, although I am not sure if it is easily done. We also have the first partial derivative test at hand, so by checking the signs of $\frac{\partial\ell}{\partial\theta}$, we can conclude regarding point of maxima. Sep 30, 2018 at 8:40

Since $$\theta >0$$ it suffices to show the numerator is always negative. Note that you only need to show it's negative at the MLE, not all values of $$\theta$$.

$$-3\sum x_i^2 + 2 \sum x_i \theta <0$$

Substitute in the MLE to obtain

$$-3 \sum_i x_i^2 + 2\sum_i x_i \frac{\sum_j x_j^2}{\sum_j x_j}$$

Cancel off the two sums on the right hand term

$$=-3\sum_i x_i^2 + 2 \sum_j x_j^2$$

$$=-1 \sum_i x_i^2 <0$$

Which is negative since every summand $$x_i^2 >0$$

Hence since the second derivative is negative at $$\theta^{MLE}$$, it's a local maximum of the likelihood function.

As pointed out by @StubbonAtom your derived MLE is incorrect. For example, examine the following R code with $$\theta = 2$$, the estimator you derived gives $$\hat{\theta} \approx 4$$.

x<- rnorm(10000,2,2)
sum(x^2)/sum(x)
[1] 4.009042