If $X$ and $Y$ are uncorrelated random variables, then under what condition is $E[X \mid Y] \approx E[X]?$

Question

Suppose $X$ and $Y$ are real random variables that are uncorrelated. Now, uncorrelated does not imply independence, so $E[X \mid Y] \ne E[X]$.

However, can they be said to be approximately equal? If so, under what conditions does that approximation hold? (I realize that the equality is exact when $X,Y$ is a multivariate Gaussian, but I want to know more generally when the equality can be approximated.)

As a bonus, is it possible to do an expansion of $E[X \mid Y]$ that looks like "$E[X] + $higher order terms", so we can see explicitly when those higher order terms can be said to be negligible? Let's assume that $p(X \cap Y)$ is continuous and infinitely differentiable.

Answer 1

One situation where this is interestingly almost true is when $X$ and $Y$ are both projections from the same high-dimensional distribution, ie, $X=a^TZ$, $Y=b^TZ$ for high-dimensional $Z$. Hall & Li (and earlier work by various people) showed that for 'most' distributions for $Z$ on a high-dimensional sphere and most $a,b$, $E[X|Y]$ is approximately linear in Y and so if they are uncorrelated they are close to independent.

The result makes sense, because $X$ and $Y$ will be approximately bivariate Gaussian by the CLT, but actually pinning down the error bounds takes work.

This question was motivated by the sliced inverse regression method of Duan and Li, where you regress $X$ on $Y$ to learn about $Y|X$

Answer 2

No, they cannot be said to be approximately equal in general unless they are exactly equal. To see this, consider:

$$\mathbb{E}[X|Y] - \mathbb{E}[X] = \delta$$

for any $\delta$ that is near the boundary of what you consider to be "approximately equal". Now, multiply $X$ by $10^9$:

$$\mathbb{E}[10^9X|Y] - \mathbb{E}[10^9X] = 10^9\mathbb{E}[X|Y] - 10^9\mathbb{E}[X] = 10^9\delta$$

and now $\delta$ is no longer in the range of "approximately equal" values.

You should be able to see this also prevents the expansion of $E[X|Y]$ from giving you any useful information in this regard; basically, one way or another, you'd probably need to actually compute the conditional and unconditional expectations and compare them to determine whether the difference is ignorable in your application, or perhaps compute bounds on the difference and use those as a decision tool instead.

Answer 3

Now, uncorrelated does not imply independence, so $E[X \mid Y] \ne E[X]$.

I find the conclusion 'so $E[X \mid Y] \ne E[X]$' in this sentence a bit confusing.

1. If variables are uncorrelated then it does not follow $E[X \mid Y] \ne E[X]$.
2. In addition independence does neither mean $E[X \mid Y] \ne E[X]$. You can have independence while $E[X \mid Y] = E[X]$

---

If you have zero correlation then you can still have dependence, and also while $E[X \vert Y] = E[X]$

Example: Let $X \sim N(0,1)$ and $Y = N(0,\sigma^2 = X^2)$

---

If you have zero correlation then

• then this means that you have a slope of zero for a line that fits $E[Y|X]$ as function of $X$ or $E[X|Y]$ as function of $Y$.

• But $E[Y|X]$ can have all sorts of deviations from the straight line.

Example: let $X \sim N(0,1)$ and $Z = N(X^2,1)$

---

However, can they be said to be approximately equal?

In many situations, you have zero correlation, but still dependence due to heterogeneity as in the first example. There can be dependence but still $E[Y|X] = E[Y]$.

But it is difficult to give general conditions for this. The condition for $E[Y|X] = E[Y]$ is that $E[Y|X] = E[Y]$.