1
$\begingroup$

I'm working through a homework assignment and I'm having an inordinately difficult time trying to calculate the MSE of two different estimators. The homework is described as this: Let $X$ and $Y$ be random variables with joint density:

$p(x,y) = 2$ if $x\in A$

It is $0$ otherwise.

$A$ is a set surrounded by $X = 0, Y = 0$, and $X + Y = 1$.

I have to calculate the MSE of 2 estimators: \begin{equation} \hat{X} = E[X|Y] \end{equation} and \begin{equation} \hat{X} = Y \end{equation}

I've seen the proof that $E[X|Y]$ is the better estimator, but I am absolutely stumped on how to actually calculate the MSE $E[(\hat X - X)^2]$ for either estimator. I've spent hours reading and trying to figure this out but I have almost no experience with statistics and it is killing me in this problem. Any help would be appreciated.

Edit:

$E[X|Y] = \int_0^1 \int_0^{1-y} 2xy dx dy = 1/12$

$E[Y] = \int_0^1 2ydy = 1$

$\endgroup$
4
  • $\begingroup$ Have you calculated $E[X|Y]$? $\endgroup$ Nov 9, 2013 at 0:31
  • $\begingroup$ I think so: \begin{equation}\int_0^1 \int_0_(1-Y) 2xy dx dy\end{equation} = 1/12 It turns out I'm not great at using LaTeX. The value is Integral(0->1){Integral(0->1-Y){ 2xy dx}dy} = 1/12 (I've done the math on paper and it worked out to this) $\endgroup$ Nov 9, 2013 at 0:44
  • $\begingroup$ Did you mean $\int_0^1 \int_0^{(1-y)} 2xy \,dx \,dy\,\,$? $\endgroup$
    – Glen_b
    Nov 9, 2013 at 0:52
  • $\begingroup$ That is what I meant, yes. $\endgroup$ Nov 9, 2013 at 0:54

1 Answer 1

1
$\begingroup$

In this problem, the conditional density of $X$ given the value of $Y$ is $\alpha$ is a uniform density on $[0,1-\alpha]$ and thus has mean $\frac{1}{2}(1-\alpha)$. Thus, $\hat{X} = E[X\mid Y] = \frac{1}{2}(1-Y)$. Can you calculate $E[(\hat{X}-X)^2]$ from this?

Added Note:
$\displaystyle E[(\hat{X}-X)^2] = E\left[\left(\frac{1}{2}(1-Y) - X\right)^2\right] = \int_{y=0}^1 \int_{x=0}^{1-y}\left(\frac{1}{2}(1-y) - x\right)^2\cdot 2 \,\mathrm dx \,\mathrm dy$

Further note in response to the OP's comments:

An easier way of calculating the value of $E[(\hat{X}-X)^2]$ is to note that conditioned on $Y = \alpha$, the conditional pdf of $X$ is a uniform density support $[0, 1-\alpha]$ and mean $\hat{X} = \frac{1}{2}(1-\alpha)$, so that $$E\left[(\hat{X}-X)^2\mid Y = \alpha\right] = \frac{(1-\alpha)^2}{12},$$ from which we get $E\left[(\hat{X}-X)^2\mid Y\right] = \frac{(1-Y)^2}{12}$ and $$E[(\hat{X}-X)^2] = E\left[E\left[(\hat{X}-X)^2\mid Y\right]\right] = E\left[\frac{(1-Y)^2}{12}\right] = \int_0^1 \frac{(1-\alpha)^2}{12}\cdot 2(1-\alpha)\,\mathrm d\alpha = \frac{1}{24}$$

$\endgroup$
6
  • $\begingroup$ Okay, so $E[(\hat X - X)^2] = Var(\hat X) + Bias(\hat X, X)^2$ Because $\hat X = E[X|Y]$, and $Bias(\hat X, X) = E[\hat X] - E[X]$, then $Bias(\hat X, X) = E[E[X|Y]] - E[X]$, and by the law of iterated expectations this becomes $E[X] - E[X] = 0$. It follows, then, that $E[(\hat X - X)^2] = Var(\hat X)$. $Var(\hat X) = E[\hat {X}^2] - (E[\hat X])^2$. By my calculations $E[\hat {X}^2] = 1/48$ and $(E[\hat X])^2 = 1/144$, which means $E[(\hat X - X)^2] = Var(\hat X) = 1/72$. That's what I have, is it wrong? $\endgroup$ Nov 9, 2013 at 20:42
  • $\begingroup$ Also, as a part of the problem I (think) I calculated $E[(\hat X - X)^2]$ for $\hat X = Y$. The estimator is unbiased according to my work, and this makes sense because X and Y are uniformly distributed over the same set which means that the error is based on the variance. Then, $Var(Y) = E[Y^2] - (E[Y])^2$ where $E[Y^2] = \int_0^1 y^2p_{y}(y)dy = \int_0^1 y^2(2-2y)dy = \int_0^1 2y^2 - 2y^3 dy = [(2/3)y^3-(1/2)y^4]_0^1 = 1/6$ and $(E[Y])^2 = ((1/2)\int_0^1 1-x dx)^2 = ((1/2)[x-(1/2)x^2]_0^1)^2 = ([1-1/2]*(1/2))^2 = (1/4)^2 = 1/16$ $\endgroup$ Nov 9, 2013 at 22:28
  • $\begingroup$ $(E[Y])^2 = (\frac{1}{2}\int_0^1 1-x dx)^2 = (\frac{1}{2}[x-\frac{x^2}{2}]_0^1)^2 = ([1-\frac{1}{2}]*\frac{1}{2})^2 = (\frac{1}{4})^2 = \frac{1}{16}$ So, $Var(Y) = \frac{1}{6} - \frac{1}{16} = \frac{5}{48}$. As a result, we can compare the MSE for the estimators $\hat X_1 = E[X|Y]$ and $\hat X_2 = Y$ and see that $\hat X_1 < \hat X_2$ which makes sense given that the conditional expectation is the best predictor. $\endgroup$ Nov 9, 2013 at 22:39
  • $\begingroup$ I evaluated the integral (using a bunch of substitutions) and I got that $E[(\hat X - X)^2] = \frac{1}{24}$ $\endgroup$ Nov 9, 2013 at 23:42
  • $\begingroup$ $\frac{1}{24}$ is the correct answer. $\endgroup$ Nov 9, 2013 at 23:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.