5
$\begingroup$

I found this formula written on a blackboard entering a math class

$$E[X|Y]= \frac{E[XY]Y}{E[Y^2]} $$

At first glance, this seemed too good to be true to me, but it could be due to my ignorance.

I have not seen this before and was wondering if this was true or under which conditions is it true?

$\endgroup$
5
$\begingroup$

Here's one case: $X,Y$ are bivariate Normal, each with mean $0$.

\begin{align*} E[X \mid Y] &= EX + \rho \frac{\sigma_X}{\sigma_Y}[Y - EY] \\ &= \frac{E[XY]}{EY^2 - [EY]^2}Y\\ &= \frac{E[XY]Y}{E[Y^2]}. \end{align*}

$\endgroup$
5
$\begingroup$

$E[X \mid Y]$ is a random variable that is a function of the random variable $Y$ (not $X$), say $g(Y)$ and it is the minimum mean-square error (MMSE) estimator of $X$ in terms of $Y$. That is, $g(\cdot)$ is the (measurable) function that minimizes the mean-square error $E[(X-h(Y))^2]$ over all choices of (measurable) functions $h(\cdot)$ that we might use to estimate $X$ in terms of $Y$. Thus, the property in question $$ E[X \mid Y] = \frac{E[XY]Y}{E[Y^2]} = \left(\frac{E[XY]}{E[Y^2]}\right)Y\tag{1}$$ holds whenever the function $g(\cdot)$ is just a scalar multiple of $Y$. Now, if we restrict the functions $h(y)$ to be linear functions (more correctly, affine functions) of the form $ay+b$, then the function $g_L(Y)$ that minimizes $E[(X-h(Y))^2]$ over all choices of linear functions $h(\cdot)$ is the linear MMSE estimator of $X$ in terms of $Y$ and is known to be $$E[X] + \left.\left.\rho\frac{\sigma_X}{\sigma_Y}\right(Y - E[Y]\right) = E[X] + \left.\left.\frac{\operatorname{cov}(X,Y)}{\operatorname{var}(Y)}\right(Y - E[Y]\right).\tag{2}$$ Thus, $(1)$ holds when the MMSE estimate of $X$ given $Y$ is proportional to $Y$ (is a linear estimate) and so is the same as the linear MMSE estimate. But, $(1)$ has no constant term as in $(2)$ and so it must be that $$E[X] = \frac{\operatorname{cov}(X,Y)}{\operatorname{var}(Y)}E[Y]\tag{3}$$ to make the constant term in $(2)$ vanish. But more is needed since whenever $(3)$ holds, $(2)$ becomes $$\left(\frac{\operatorname{cov}(X,Y)}{\operatorname{var}(Y)}\right)Y = \left(\frac{E[XY]-E[X]E[Y]}{E[Y^2]-(E[Y])^2}\right)Y \neq \left(\frac{E[XY]}{E[Y^2]}\right)Y $$ unless $E[X]=E[Y]=0$ (and so $(3)$ continues to be satisfied). In short, $(1)$ holds whenever

  • $X$ and $Y$ are zero-mean random variables

and

  • $E[X\mid Y]$, the MMSE estimator of $X$ given $Y$ is a linear estimator $E[X\mid Y] = aY$.

A standard example when these conditions hold is when $X$ and $Y$ are zero-mean random variables enjoying a bivariate normal distribution as in @Taylor's answer, but this is by no means the only possible example. Consider $(X,Y)$ uniformly distributed on the interior or the parallelogram with vertices $(-1,-1)$, $(0,-1)$, $(1,1)$ and $(0,1)$. It is easily verified that $E[X] = E[Y] = 0$. Note also that for any $y \in (-1,1)$, the conditional distribution of $X$ given that $Y = y$ is $\mathcal U\left(-\frac 12 + \frac y2, \frac 12 + \frac y2\right)$ and so $E[X\mid Y] = Y$. Verification that $E[XY] = E[Y^2]$ is left as an exercise for the skeptical reader.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.