# $N(\theta,\theta)$: MLE for a Normal where mean=variance

$\newcommand{\nd}{\frac{n}{2}}$For an $n$-sample following a Normal$(\mu=\theta,\sigma^2=\theta)$, how do we find the mle?

I can find the root of the score function $$\theta=\frac{1\pm\sqrt{1-4\frac{s}{n}}}{2},s=\sum x_i^2,$$ but I don't see which one is the maximum.
I tried to substitute in the second derivative of the log-likelihood, without success.

For the likelihood, with $x=(x_1,x_2,\ldots,x_n)$, $$f(x) = (2\pi)^{-n/2} \theta^{-n/2} \exp\left( -\frac{1}{2\theta}\sum(x_i-\theta)^2\right),$$ then, with $s=\sum x_i^2$ and $t=\sum x_i$, $$\ln f(x) = -\nd \ln(2\pi) -\nd\ln\theta-\frac{s}{2\theta}-t+\nd\theta,$$ so that $$\partial_\theta \ln f(x) = -\nd\frac{1}{\theta}+\frac{s}{2\theta^2}+\nd,$$ and the roots are given by $$\theta^2-\theta+\frac{s}{n}=0.$$ Also, $$\partial_{\theta,\theta} \ln f(x) = \nd \frac{1}{\theta^2} - \frac{s}{\theta^3}.$$

• It looks to me like there might be an error in your calculation of $\log f(x)$. I think it should be $\mbox{const} -\frac{n}{2} \log(\theta) - \frac{s}{2\theta} + t - \frac{n \theta}{2}$. As is, there is a positive probability chance that $1 - 4 \frac s n < 0$ which is a problem. – guy Apr 16 '13 at 19:57
• Think about the meaning. Why do you want to take the derivative? You want to maximize the likelihood function. Is there a necessary condition for a maximum point of a differentiable function? – Preston Lui Jul 3 at 15:43

There are some typos (or algebraical mistakes) in the signs of the log-likelihood, followed by the corresponding unpleasant consequences.

Since this is a well-known problem, I will only point out a reference with the solution:

Asymptotic Theory of Statistics and Probability pp. 53, by Anirban DasGupta.

Recall that the normal distribution $$N(\mu, \sigma^2)$$ has pdf $$f(x\mid \mu ,\sigma ^{2})={\frac {1}{{\sqrt {2\pi \sigma ^{2}}}\ }}\exp {\left(-{\frac {(x-\mu )^{2}}{2\sigma ^{2}}}\right)},$$ Note here that $$\mu = \theta$$ and $$\sigma^2 = \theta$$ and therefore $$\sigma = \sqrt{\theta}$$

\begin{aligned} L(x_1,x_2,...,x_n | \theta) &= \prod_{i=1}^n f(x_i | \theta) \\ &= \prod_{i=1}^n \frac{1}{\sqrt{2 \pi \theta}} \ \exp \Big \{ - \frac{1}{2 \theta} (x_i - \theta)^2 \Big\} \\ & = (2 \pi)^{-n/2} (\theta)^{-n /2} \prod_{i=1}^n \ \exp \Big \{ - \frac{1}{2 \theta} (x_i - \theta)^2 \Big\} \\ & = (2 \pi)^{-n/2} (\theta)^{-n /2} \ \exp \Big \{ - \frac{1}{2 \theta} \sum_{i=1}^n (x_i - \theta)^2 \Big\} \\ \log L& = - \frac{n}{2} \log(2\pi) - \frac{n}{2} \log(\theta) - \frac{1}{2 \theta} \sum_{i=1}^n (x_i - \theta)^2 \end{aligned}

Consider the term $$\frac{1}{2 \theta} \sum_{i=1}^n (x_i - \theta)^2$$ which can be expanded and simplified \begin{aligned} \frac{1}{2 \theta} \sum_{i=1}^n (x_i - \theta)^2 & = \frac{1}{2 \theta} \sum_{i=1}^n (x_i - \theta)(x_i - \theta) \\ & = \frac{1}{2 \theta} \sum_{i=1}^n \left( x_i^2 - 2 \theta x_i + \theta^2 \right) \\ & = \frac{1}{2 \theta} \left( \sum_{i=1}^n (x_i^2) - 2 \theta \sum_{i=1}^n (x_i) + n\theta^2 \right) \\ & = \frac{1}{2 \theta} \sum_{i=1}^n (x_i^2) - \sum_{i=1} (x_i) + \frac{n\theta}{2} \end{aligned}

We can now compute the derivative with respect to $$\theta$$, equate to zero and solve for $$\theta$$

\begin{aligned} \log L& = - \frac{n}{2} \log(2\pi) - \frac{n}{2} \log(\theta) - \left( \frac{1}{2 \theta} \sum_{i=1}^n (x_i^2) - \sum_{i=1} (x_i) + \frac{n\theta}{2} \right) \\ \frac{d}{d\theta} \log L & = \frac{-n}{2\theta} - \left( \frac{-1}{2\theta^2} \sum_{i=1}^n (x_i^2) + \frac{n}{2} \right) = 0 \\ & = \frac{-n}{2\theta} + \frac{1}{2\theta^2} \sum_{i=1}^n (x_i^2) - \frac{n}{2} \\ & = - \theta^2 - \theta + \frac{1}{n} \sum_{i=1}^n (x_i^2) \\ &\text{let s = \frac{1}{n} \sum_{i=1}^n (x_i^2)} \\ 0 & = - \theta^2 - \theta + s \\ \hat \theta &= \frac{\sqrt{1 + 4s} -1 }{2} \end{aligned}

Consider $$\log f(x) = -0.5\log (2 \pi \theta) - 0.5 \frac{(x - \theta)^2}{\theta}$$ and $$\frac{\partial}{\partial\theta} \log f(x) \propto -\frac{1}{\theta}+\frac{x^2}{\theta^2} -1$$ Thus, $$\frac{\partial}{\partial\theta} \ell (x) = 0 = -n(1 + \frac{1}{\theta}) +\sum \frac{x_k^2}{\theta^2}$$ so $$\theta^2 + \theta = \frac{1}{n}\sum x_k^2$$ which gives

$$\theta^* = \sqrt{\frac{1}{n}\sum x_k^2 + \tfrac{1}{4}}-\frac{1}{2}$$.

Ignore the negative root since it contradicts $$\theta \ge0$$.