Find the MLE $\hat\theta$ for $\theta$ [closed]

I am trying to learn by myself, this is the problem I came across.

If $$X_1, X_2 ..... X_n$$ be an i.i.d. sample from $$N(\theta, \theta^2)$$ distribution, where $$\theta > 0$$ is unknown parameter.

What is the MLE $$\hat\theta$$ of $$\theta$$? And how do I show that MLE is consistent for $$\theta$$?

• Welcome. Where are you stuck? What have you tried?
– Ale
Dec 10, 2020 at 10:58

This is pretty straightforward. The PDF of each $$X_i$$ is : $$f_{X_i} (x) = \dfrac{1}{\theta\sqrt{2\pi}}\cdot \exp\left[-\dfrac{1}{2}\left(\dfrac{x - \theta}{\theta}\right)^2\right]\quad\quad\text{for}~i=1, 2, \cdots, n$$
Thus, the likelihood equation will be : $$L = \prod_{i=1}^n \dfrac{1}{\theta\sqrt{2\pi}}\cdot \exp\left[-\dfrac{1}{2}\left(\dfrac{x_i - \theta}{\theta}\right)^2\right] = \dfrac{1}{\theta^n (2\pi)^\frac{n}{2}}\cdot \exp\left[-\dfrac{1}{2}\sum_{i=1}^n \left(\dfrac{x}{\theta} - 1 \right)^2\right]$$ Therefore, $$-\log L = n \log \theta +\dfrac{n}{2} \log (2\pi) + \dfrac12 \displaystyle\sum_{i=1}^n \left(\dfrac{x}{\theta} - 1 \right)^2$$ In order to maximize $$L$$ , we must have $$\dfrac{dL}{d\theta} = 0$$ . Differentiating $$L$$ , equating to $$0$$ , we get : $$\dfrac{n}{\theta} - \sum_{i=1}^n \left(\dfrac{x}{\theta} - 1 \right)\cdot \dfrac{x_i}{\theta^2} = 0\quad~\implies~\quad \left(\dfrac{1}{\theta} \right)^2\cdot \sum_{i=1}^n x_i ^2 - \left(\dfrac{1}{\theta}\right) \cdot \sum_{i=1}^n x_i - n = 0$$ Solving this quadratic equation, we get that : $$\text{MLE}(\theta) ~=~ \hat{\theta} ~=~ \dfrac{2 \cdot \displaystyle\sum_{i=1}^n x_i^2}{\displaystyle\sum_{i=1}^n x_i + \sqrt{\left(\sum_{i=1}^n x_i \right)^2 + 4n\cdot \sum_{i=1}^n x_i^2} }$$
• Presumably the quadratic has two solutions (why does you final expression not include $n$?) and you may need to check which if either gives a maximum: for example I would expect something strange to happen in some cases if $\sum x_i < 0$ Dec 10, 2020 at 11:32
• My final expression had a small typo, I fixed it almost immediately after writing my answer. The other root of the quadratic is not a correct answer because $\theta$ would have been negative in that case. Dec 10, 2020 at 14:50
• @Henry yes, we should check whether the 2nd derivative is positive or not. I have omitted that in my solution. However, in most cases, I have seen that the equation which we get after setting $\frac{dL}{d\theta} = 0$ produces a solution which actually maximizes the log-likelihood (and not minimizes it). I don't know why this happens. It'll be helpful if you can say why $\hat{\theta}$ maximizes $L$, without calculating the 2nd derivative. Dec 10, 2020 at 15:00
• I see your point. If $n=2, x_1=-1,x_2=-2$ then your expression gives $\hat \theta=2.5$ and a quick curve sketch shows this strange result is indeed the maximum of the likelihood Dec 10, 2020 at 15:09