0
$\begingroup$

I am trying to learn by myself, this is the problem I came across.

If $X_1, X_2 ..... X_n$ be an i.i.d. sample from $N(\theta, \theta^2)$ distribution, where $\theta > 0$ is unknown parameter.

What is the MLE $\hat\theta$ of $\theta$? And how do I show that MLE is consistent for $\theta$?

Improve this question
$\endgroup$
1
  • 2
    $\begingroup$ Welcome. Where are you stuck? What have you tried? $\endgroup$
    – Ale
    Dec 10, 2020 at 10:58

1 Answer 1

Reset to default
3
$\begingroup$

This is pretty straightforward. The PDF of each $X_i$ is : $$f_{X_i} (x) = \dfrac{1}{\theta\sqrt{2\pi}}\cdot \exp\left[-\dfrac{1}{2}\left(\dfrac{x - \theta}{\theta}\right)^2\right]\quad\quad\text{for}~i=1, 2, \cdots, n$$

Thus, the likelihood equation will be : $$L = \prod_{i=1}^n \dfrac{1}{\theta\sqrt{2\pi}}\cdot \exp\left[-\dfrac{1}{2}\left(\dfrac{x_i - \theta}{\theta}\right)^2\right] = \dfrac{1}{\theta^n (2\pi)^\frac{n}{2}}\cdot \exp\left[-\dfrac{1}{2}\sum_{i=1}^n \left(\dfrac{x}{\theta} - 1 \right)^2\right]$$ Therefore, $$-\log L = n \log \theta +\dfrac{n}{2} \log (2\pi) + \dfrac12 \displaystyle\sum_{i=1}^n \left(\dfrac{x}{\theta} - 1 \right)^2$$ In order to maximize $L$ , we must have $\dfrac{dL}{d\theta} = 0$ . Differentiating $L$ , equating to $0$ , we get : $$\dfrac{n}{\theta} - \sum_{i=1}^n \left(\dfrac{x}{\theta} - 1 \right)\cdot \dfrac{x_i}{\theta^2} = 0\quad~\implies~\quad \left(\dfrac{1}{\theta} \right)^2\cdot \sum_{i=1}^n x_i ^2 - \left(\dfrac{1}{\theta}\right) \cdot \sum_{i=1}^n x_i - n = 0$$ Solving this quadratic equation, we get that : $$\text{MLE}(\theta) ~=~ \hat{\theta} ~=~ \dfrac{2 \cdot \displaystyle\sum_{i=1}^n x_i^2}{\displaystyle\sum_{i=1}^n x_i + \sqrt{\left(\sum_{i=1}^n x_i \right)^2 + 4n\cdot \sum_{i=1}^n x_i^2} }$$

Check my calculation once. I may be wrong somewhere.

Improve this answer
$\endgroup$
5
  • $\begingroup$ Presumably the quadratic has two solutions (why does you final expression not include $n$?) and you may need to check which if either gives a maximum: for example I would expect something strange to happen in some cases if $\sum x_i < 0$ $\endgroup$
    – Henry
    Dec 10, 2020 at 11:32
  • $\begingroup$ My final expression had a small typo, I fixed it almost immediately after writing my answer. The other root of the quadratic is not a correct answer because $\theta$ would have been negative in that case. $\endgroup$
    – Kolmogorov
    Dec 10, 2020 at 14:50
  • $\begingroup$ @Henry yes, we should check whether the 2nd derivative is positive or not. I have omitted that in my solution. However, in most cases, I have seen that the equation which we get after setting $\frac{dL}{d\theta} = 0$ produces a solution which actually maximizes the log-likelihood (and not minimizes it). I don't know why this happens. It'll be helpful if you can say why $\hat{\theta}$ maximizes $L$, without calculating the 2nd derivative. $\endgroup$
    – Kolmogorov
    Dec 10, 2020 at 15:00
  • $\begingroup$ I see your point. If $n=2, x_1=-1,x_2=-2$ then your expression gives $\hat \theta=2.5$ and a quick curve sketch shows this strange result is indeed the maximum of the likelihood $\endgroup$
    – Henry
    Dec 10, 2020 at 15:09
  • $\begingroup$ Intuitively that's clear. However, that's surely not a rigorous thing. Anyway, it serves our purpose. $\endgroup$
    – Kolmogorov
    Dec 10, 2020 at 15:22

Not the answer you're looking for? Browse other questions tagged or ask your own question.