8
$\begingroup$

Suppose $X$ and $Y$ are real random variables that are uncorrelated. Now, uncorrelated does not imply independence, so $E[X \mid Y] \ne E[X]$.

However, can they be said to be approximately equal? If so, under what conditions does that approximation hold? (I realize that the equality is exact when $X,Y$ is a multivariate Gaussian, but I want to know more generally when the equality can be approximated.)

As a bonus, is it possible to do an expansion of $E[X \mid Y]$ that looks like "$E[X] + $higher order terms", so we can see explicitly when those higher order terms can be said to be negligible? Let's assume that $p(X \cap Y)$ is continuous and infinitely differentiable.

$\endgroup$
6
  • $\begingroup$ Correlation tells you absolutely nothing about expectations. $\endgroup$
    – whuber
    Oct 20, 2021 at 19:27
  • $\begingroup$ @whuber Are you claiming that they are mutually independent? Or that no proven propositions/theorems/lemmas come to mind? Or something else? $\endgroup$
    – Galen
    Oct 20, 2021 at 21:54
  • $\begingroup$ Consider $X=(Y-\mu_Y)^2$ for $Y$ symmetrically distributed with nonzero variance and finite third moment for one example. $\text{Cov}(X,Y) = 0$ but $E(Y|X)$ could be very different from $E(Y)$. Five of the ten examples here: i.stack.imgur.com/Akcli.png show various of the many ways that $E(Y|X)$ may potentially differ from $E(Y)$ -- in some cases by a lot. Indeed it may well be the case that the two are never close to equal for any $x$ $\endgroup$
    – Glen_b
    Oct 20, 2021 at 22:20
  • $\begingroup$ @Galen Correlation gives zero information about expectations. Proof: the correlation of $(X,Y)$ and the correlation of $(X+\mu,Y)$ are the same for all constants $\mu.$ $\endgroup$
    – whuber
    Oct 20, 2021 at 22:21
  • 1
    $\begingroup$ @whuber I see you what you mean. Yes, correlation is shift invariant while expectations are not. Thank you for clarifying what you were getting at. $\endgroup$
    – Galen
    Oct 20, 2021 at 22:33

3 Answers 3

2
$\begingroup$

No, they cannot be said to be approximately equal in general unless they are exactly equal. To see this, consider:

$$\mathbb{E}[X|Y] - \mathbb{E}[X] = \delta$$

for any $\delta$ that is near the boundary of what you consider to be "approximately equal". Now, multiply $X$ by $10^9$:

$$\mathbb{E}[10^9X|Y] - \mathbb{E}[10^9X] = 10^9\mathbb{E}[X|Y] - 10^9\mathbb{E}[X] = 10^9\delta$$

and now $\delta$ is no longer in the range of "approximately equal" values.

You should be able to see this also prevents the expansion of $E[X|Y]$ from giving you any useful information in this regard; basically, one way or another, you'd probably need to actually compute the conditional and unconditional expectations and compare them to determine whether the difference is ignorable in your application, or perhaps compute bounds on the difference and use those as a decision tool instead.

$\endgroup$
3
  • $\begingroup$ Thanks for the answer. But when I say "under what conditions", I don't only mean conditions on the probability distribution that allows the approximation to apply for all values of $X$. I would also accept conditions on the region of $X$ itself, such as "for small $|X|$...". But I'll upvote because this is a good answer to the former specification of "conditions". $\endgroup$ Oct 20, 2021 at 19:08
  • $\begingroup$ For example, it's true for Gaussian distribution that $E[X \mid Y] = E[X]$ if they are uncorrelated. Intuitively, I feel like if there was some distribution that was close to a Gaussian in form (like, a distribution whose expansion is equal to a Gaussian in first order, but diverges for large X,Y) then the equality would be close to true, but would fall off quickly as $X$ or $Y$ gets large. Is my intuition right, and if so, could we state something like that formally? $\endgroup$ Oct 20, 2021 at 19:21
  • $\begingroup$ Hmmm... that's a much more interesting explanation of your question than I had previously understood it to be. I may wind up deleting my answer, which doesn't address your actual question. Could you put a statement like that in the question itself, so new readers don't miss it? $\endgroup$
    – jbowman
    Oct 20, 2021 at 21:04
2
$\begingroup$

One situation where this is interestingly almost true is when $X$ and $Y$ are both projections from the same high-dimensional distribution, ie, $X=a^TZ$, $Y=b^TZ$ for high-dimensional $Z$. Hall & Li (and earlier work by various people) showed that for 'most' distributions for $Z$ on a high-dimensional sphere and most $a,b$, $E[X|Y]$ is approximately linear in Y and so if they are uncorrelated they are close to independent.

The result makes sense, because $X$ and $Y$ will be approximately bivariate Gaussian by the CLT, but actually pinning down the error bounds takes work.

This question was motivated by the sliced inverse regression method of Duan and Li, where you regress $X$ on $Y$ to learn about $Y|X$

$\endgroup$
1
$\begingroup$

Now, uncorrelated does not imply independence, so $E[X \mid Y] \ne E[X]$.

I find the conclusion 'so $E[X \mid Y] \ne E[X]$' in this sentence a bit confusing.

  1. If variables are uncorrelated then it does not follow $E[X \mid Y] \ne E[X]$.
  2. In addition independence does neither mean $E[X \mid Y] \ne E[X]$. You can have independence while $E[X \mid Y] = E[X]$

If you have zero correlation then you can still have dependence, and also while $E[X \vert Y] = E[X]$

Example: Let $X \sim N(0,1)$ and $Y = N(0,\sigma^2 = X^2)$

example


If you have zero correlation then

  • then this means that you have a slope of zero for a line that fits $E[Y|X]$ as function of $X$ or $E[X|Y]$ as function of $Y$.

  • But $E[Y|X]$ can have all sorts of deviations from the straight line.

Example: let $X \sim N(0,1)$ and $Z = N(X^2,1)$

example


However, can they be said to be approximately equal?

In many situations, you have zero correlation, but still dependence due to heterogeneity as in the first example. There can be dependence but still $E[Y|X] = E[Y]$.

But it is difficult to give general conditions for this. The condition for $E[Y|X] = E[Y]$ is that $E[Y|X] = E[Y]$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.