3
$\begingroup$

Suppose $X_1, X_2, \dots, X_n$ are i.i.d $N(\theta, 1),\theta_0 \le \theta \le \theta_1$, where $\theta_0 \lt \theta_1$ are two specified numbers. Find the MLE of $\theta$ and show that it is better than the sample mean $\bar X$ in the sense of having smaller mean squared error.

Trial: Here MLE of $\theta$ is $$\hat\theta = \begin{cases} \theta_0, &\bar X \lt \theta_0\\ \bar X, &\theta_0 \le \bar X \le \theta_1 \\ \theta_1, &\bar X \gt \theta_1 \end{cases}$$

I can't show that $E[(\hat\theta -\theta)^2] \lt Var(\bar X)=\dfrac{1}{n}$.

$\endgroup$
2
  • 1
    $\begingroup$ First of all, get your notation correct. You are asked to show that $E[(\hat{\theta}-\theta)^2] < \operatorname{var}(\bar{X})$ and not $E[\hat\theta -\theta]^2$ as you have it. Your expression would be taken to mean $\left(E[\hat\theta -\theta]\right)^2$ the square of the mean, whereas what is wanted is the mean of the square. Then argue that the difference between the distributions of $\bar{X}$ and $\hat{\theta}$ is that all the probability mass to the left of $\theta_0$ (to the right of $\theta_1$) has been replaced by a point mass at $\theta_0$ (at $\theta_1$) and this reduces the variance. $\endgroup$ Commented May 10, 2013 at 11:58
  • 1
    $\begingroup$ @DilipSarwate unfortunately some authors actually use such an ambiguous notation. I applaud your encouragement of its avoidance. $\endgroup$
    – Glen_b
    Commented May 11, 2013 at 6:36

1 Answer 1

4
$\begingroup$

I assume that you already know that $\operatorname{var}(\bar{X}) = \frac{1}{n}$ or can prove this result if it is also asked for, and also that you know that $E[\bar{X}] = \theta$.

Expanding on my comment, let $g(x)$ denote the probability density function of $\bar{X}$. Then, $$\begin{align} \operatorname{var}(\bar{X}) &= \int_{-\infty}^\infty (x-\theta)^2g(x)\,\mathrm dx\\ &= \int_{-\infty}^{\theta_0} (x-\theta)^2g(x)\,\mathrm dx + \int_{\theta_0}^{\theta_1}(x-\theta)^2g(x)\,\mathrm dx + \int_{\theta_1}^\infty (x-\theta)^2g(x)\,\mathrm dx\\ &> (\theta_0-\theta)^2\int_{-\infty}^{\theta_0}g(x)\,\mathrm dx + \int_{\theta_0}^{\theta_1}(x-\theta)^2g(x)\,\mathrm dx + (\theta_1-\theta)^2\int_{\theta_1}^\infty g(x)\,\mathrm dx\\ &= (\theta_0-\theta)^2P\{\hat{\theta}=\theta_0\} + \int_{\theta_0}^{\theta_1}(x-\theta)^2g(x)\,\mathrm dx + (\theta_0-\theta)^2P\{\hat{\theta}=\theta_1\}\\ &= E[(\hat{\theta}-\theta)^2]. \end{align}$$ I will leave it to you to figure out how the inequality in the third line comes about, how two integrals become probabilities involving $\hat{\theta}$ in the fourth line, and why the sum on the fourth line works out to be $E[(\hat{\theta}-\theta)^2]$ as claimed on the last line.

Incidentally, though you have tagged this with normal-distribution, the result holds as long as the i.i.d. random variables have finite variance.

$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.