2
$\begingroup$

Suppose $X_1, X_2, \dots, X_n$ are i.i.d $N(\theta, 1),\theta_0 \le \theta \le \theta_1$, where $\theta_0 \lt \theta_1$ are two specified numbers. Find the MLE of $\theta$ and show that it is better than the sample mean $\bar X$ in the sense of having smaller mean squared error.

Trial: Here MLE of $\theta$ is $$\hat\theta = \begin{cases} \theta_0, &\bar X \lt \theta_0\\ \bar X, &\theta_0 \le \bar X \le \theta_1 \\ \theta_1, &\bar X \gt \theta_1 \end{cases}$$

I can't show that $E[(\hat\theta -\theta)^2] \lt Var(\bar X)=\dfrac{1}{n}$.

$\endgroup$
  • 1
    $\begingroup$ First of all, get your notation correct. You are asked to show that $E[(\hat{\theta}-\theta)^2] < \operatorname{var}(\bar{X})$ and not $E[\hat\theta -\theta]^2$ as you have it. Your expression would be taken to mean $\left(E[\hat\theta -\theta]\right)^2$ the square of the mean, whereas what is wanted is the mean of the square. Then argue that the difference between the distributions of $\bar{X}$ and $\hat{\theta}$ is that all the probability mass to the left of $\theta_0$ (to the right of $\theta_1$) has been replaced by a point mass at $\theta_0$ (at $\theta_1$) and this reduces the variance. $\endgroup$ – Dilip Sarwate May 10 '13 at 11:58
  • 1
    $\begingroup$ @DilipSarwate unfortunately some authors actually use such an ambiguous notation. I applaud your encouragement of its avoidance. $\endgroup$ – Glen_b May 11 '13 at 6:36
4
$\begingroup$

I assume that you already know that $\operatorname{var}(\bar{X}) = \frac{1}{n}$ or can prove this result if it is also asked for, and also that you know that $E[\bar{X}] = \theta$.

Expanding on my comment, let $g(x)$ denote the probability density function of $\bar{X}$. Then, $$\begin{align} \operatorname{var}(\bar{X}) &= \int_{-\infty}^\infty (x-\theta)^2g(x)\,\mathrm dx\\ &= \int_{-\infty}^{\theta_0} (x-\theta)^2g(x)\,\mathrm dx + \int_{\theta_0}^{\theta_1}(x-\theta)^2g(x)\,\mathrm dx + \int_{\theta_1}^\infty (x-\theta)^2g(x)\,\mathrm dx\\ &> (\theta_0-\theta)^2\int_{-\infty}^{\theta_0}g(x)\,\mathrm dx + \int_{\theta_0}^{\theta_1}(x-\theta)^2g(x)\,\mathrm dx + (\theta_1-\theta)^2\int_{\theta_1}^\infty g(x)\,\mathrm dx\\ &= (\theta_0-\theta)^2P\{\hat{\theta}=\theta_0\} + \int_{\theta_0}^{\theta_1}(x-\theta)^2g(x)\,\mathrm dx + (\theta_0-\theta)^2P\{\hat{\theta}=\theta_1\}\\ &= E[(\hat{\theta}-\theta)^2]. \end{align}$$ I will leave it to you to figure out how the inequality in the third line comes about, how two integrals become probabilities involving $\hat{\theta}$ in the fourth line, and why the sum on the fourth line works out to be $E[(\hat{\theta}-\theta)^2]$ as claimed on the last line.

Incidentally, though you have tagged this with normal-distribution, the result holds as long as the i.i.d. random variables have finite variance.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.