6
$\begingroup$

I'm studying on Casella-Berger, I'm at page 322 in which it explain how to find MLE for a Gaussian distribution with parameter $\mu$ and $\sigma^2$, both unknown. It finds MLE, and up to this point it is all clear, and they are $\hat{\mu} = \bar{x}$ and $\hat{\sigma}^2 = \frac{1}{n} \sum_{i=1}^{n} (x_i-\bar{x})^2$.

Now, it says it is difficult to prove analitically that these are global maxima indeed, and it uses this fact:

If $\theta \ne \bar{x}$ then $\sum (x_i-\theta)^2 > \sum (x_i-\bar{x})^2$.

It doesn't give any explaination for that. Is there something obvious I don't see?

$\endgroup$
5
$\begingroup$

You can use a sum-of-squares argument to see this.

$$\sum_i (x_i-\theta)^2 = \sum_i (x_i - \bar{x}+\bar{x}-\theta)^2 = \sum_i (x_i-\bar{x})^2+n(\bar{x}-\theta)^2+\\\color{red}{2(\bar{x}-\theta)\sum_i(x_i-\bar{x})}$$

Now, $\bar{x}$ is defined so that the cross term becomes zero since $\sum_i x_i = \sum_i \bar{x}$.

We are therefore left with:

$$\sum_i (x_i - \theta)^2 = \sum_i (x_i-\bar{x})^2+n(\bar{x}-\theta)^2$$

This is the variance/bias decomposition of the squared-error associated with estimator $\theta$. Since both terms on the RHS are positive and only the bias term depends on $\theta$, we have:

$$\sum_i (x_i - \theta)^2 > \sum_i (x_i - \bar{x})^2$$

$\endgroup$
1
$\begingroup$

$\theta\mapsto \sum (x_i-\theta)^2$ is a quadratic function in $\theta$ that opens upwards. It has a unique minimum where its derivative is $0$, that is to say when $2\sum (x_i-\theta)=0$ ie $\theta = \hat x$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.