$N(\theta,\theta)$: MLE for a Normal where mean=variance

Question

$\newcommand{\nd}{\frac{n}{2}}$For an $n$-sample following a Normal$(\mu=\theta,\sigma^2=\theta)$, how do we find the mle?

I can find the root of the score function $$ \theta=\frac{1\pm\sqrt{1-4\frac{s}{n}}}{2},s=\sum x_i^2, $$ but I don't see which one is the maximum.
I tried to substitute in the second derivative of the log-likelihood, without success.

For the likelihood, with $x=(x_1,x_2,\ldots,x_n)$, $$ f(x) = (2\pi)^{-n/2} \theta^{-n/2} \exp\left( -\frac{1}{2\theta}\sum(x_i-\theta)^2\right), $$ then, with $s=\sum x_i^2$ and $t=\sum x_i$, $$ \ln f(x) = -\nd \ln(2\pi) -\nd\ln\theta-\frac{s}{2\theta}-t+\nd\theta, $$ so that $$ \partial_\theta \ln f(x) = -\nd\frac{1}{\theta}+\frac{s}{2\theta^2}+\nd, $$ and the roots are given by $$ \theta^2-\theta+\frac{s}{n}=0. $$ Also, $$ \partial_{\theta,\theta} \ln f(x) = \nd \frac{1}{\theta^2} - \frac{s}{\theta^3}. $$

Answer 1

There are some typos (or algebraical mistakes) in the signs of the log-likelihood, followed by the corresponding unpleasant consequences.

Since this is a well-known problem, I will only point out a reference with the solution:

Asymptotic Theory of Statistics and Probability pp. 53, by Anirban DasGupta.

Answer 2

Recall that the normal distribution $N(\mu, \sigma^2)$ has pdf $f(x\mid \mu ,\sigma ^{2})={\frac {1}{{\sqrt {2\pi \sigma ^{2}}}\ }}\exp {\left(-{\frac {(x-\mu )^{2}}{2\sigma ^{2}}}\right)},$ Note here that $\mu = \theta$ and $\sigma^2 = \theta$ and therefore $\sigma = \sqrt{\theta}$

\begin{aligned} L(x_1,x_2,...,x_n | \theta) &= \prod_{i=1}^n f(x_i | \theta) \\ &= \prod_{i=1}^n \frac{1}{\sqrt{2 \pi \theta}} \ \exp \Big \{ - \frac{1}{2 \theta} (x_i - \theta)^2 \Big\} \\ & = (2 \pi)^{-n/2} (\theta)^{-n /2} \prod_{i=1}^n \ \exp \Big \{ - \frac{1}{2 \theta} (x_i - \theta)^2 \Big\} \\ & = (2 \pi)^{-n/2} (\theta)^{-n /2} \ \exp \Big \{ - \frac{1}{2 \theta} \sum_{i=1}^n (x_i - \theta)^2 \Big\} \\ \log L& = - \frac{n}{2} \log(2\pi) - \frac{n}{2} \log(\theta) - \frac{1}{2 \theta} \sum_{i=1}^n (x_i - \theta)^2 \end{aligned}

Consider the term $\frac{1}{2 \theta} \sum_{i=1}^n (x_i - \theta)^2$ which can be expanded and simplified \begin{aligned} \frac{1}{2 \theta} \sum_{i=1}^n (x_i - \theta)^2 & = \frac{1}{2 \theta} \sum_{i=1}^n (x_i - \theta)(x_i - \theta) \\ & = \frac{1}{2 \theta} \sum_{i=1}^n \left( x_i^2 - 2 \theta x_i + \theta^2 \right) \\ & = \frac{1}{2 \theta} \left( \sum_{i=1}^n (x_i^2) - 2 \theta \sum_{i=1}^n (x_i) + n\theta^2 \right) \\ & = \frac{1}{2 \theta} \sum_{i=1}^n (x_i^2) - \sum_{i=1} (x_i) + \frac{n\theta}{2} \end{aligned}

We can now compute the derivative with respect to $\theta$, equate to zero and solve for $\theta$

\begin{aligned} \log L& = - \frac{n}{2} \log(2\pi) - \frac{n}{2} \log(\theta) - \left( \frac{1}{2 \theta} \sum_{i=1}^n (x_i^2) - \sum_{i=1} (x_i) + \frac{n\theta}{2} \right) \\ \frac{d}{d\theta} \log L & = \frac{-n}{2\theta} - \left( \frac{-1}{2\theta^2} \sum_{i=1}^n (x_i^2) + \frac{n}{2} \right) = 0 \\ & = \frac{-n}{2\theta} + \frac{1}{2\theta^2} \sum_{i=1}^n (x_i^2) - \frac{n}{2} \\ & = - \theta^2 - \theta + \frac{1}{n} \sum_{i=1}^n (x_i^2) \\ &\text{let $s = \frac{1}{n} \sum_{i=1}^n (x_i^2)$} \\ 0 & = - \theta^2 - \theta + s \\ \hat \theta &= \frac{\sqrt{1 + 4s} -1 }{2} \end{aligned}

Answer 3

Consider $\log f(x) = -0.5\log (2 \pi \theta) - 0.5 \frac{(x - \theta)^2}{\theta}$ and $$ \frac{\partial}{\partial\theta} \log f(x) \propto -\frac{1}{\theta}+\frac{x^2}{\theta^2} -1 $$ Thus, $$ \frac{\partial}{\partial\theta} \ell (x) = 0 = -n(1 + \frac{1}{\theta}) +\sum \frac{x_k^2}{\theta^2} $$ so $\theta^2 + \theta = \frac{1}{n}\sum x_k^2$ which gives

$\theta^* = \sqrt{\frac{1}{n}\sum x_k^2 + \tfrac{1}{4}}-\frac{1}{2}$.

Ignore the negative root since it contradicts $\theta \ge0$.