Is this at all possible? What is the intuition for this?


marked as duplicate by whuber Dec 4 '14 at 14:12

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

  • 2
    $\begingroup$ After writing an answer I noticed that I'd read "E(Y|X) be a nonlinear function of X" instead of "of X" - was my misreading actually the meaning that was intended? $\endgroup$ – Silverfish Dec 4 '14 at 14:11
  • 1
    $\begingroup$ The bottom right image in Cardinal's answer to the duplicate provides a clear counterexample. The answer itself gives a constructive way to create such counterexamples (via copulas), giving some intuition about the situation. NB: $E[Y|X]$ is a function of $X$, not $Y$. $\endgroup$ – whuber Dec 4 '14 at 14:15
  • $\begingroup$ See the example in the later part of the answer to this question. $\endgroup$ – Glen_b Dec 4 '14 at 16:35
  • 1
    $\begingroup$ Another dup: stats.stackexchange.com/questions/308775/… $\endgroup$ – kjetil b halvorsen Apr 28 at 20:23

This is possible, here's one example that comes to mind.

Let $X ~ \sim \mathcal{N}(0,1)$ and let $Y$ have a truncated standard normal distribution on $[0, \infty)$ if $X \geq 0$ and on $(-\infty, 0)$ if $X < 0$. Then the marginal distributions of $X$ and $Y$ are both standard normal, but $\mathbb{E}(Y|X)$ is a non-linear function of $X$.

In particular, if $X \geq 0$ then $Y$ has a half-normal distribution so has mean $\sqrt{\frac{2}{\pi}}$, and by symmetry if $X < 0$ then $Y$ has mean $-\sqrt{\frac{2}{\pi}}$. Overall $\mathbb{E}(Y|X)=\sqrt{\frac{2}{\pi}}$ for $X \geq 0$ and $-\sqrt{\frac{2}{\pi}}$ for $X < 0$ so is indeed a non-linear function of $X$.

My personal intuition for this choice was to think about "chunks of probability" of $X$ - then on each chunk, what conditional distribution could I set for $Y$ that would ensure both that (i) the marginal distribution of $Y$ is normal, (ii) the conditional mean of $Y$ is not a linear function $X$. The latter can be achieved easily if a discrete chunk of $X$'s probability distribution is taken and the same conditional distribution of $Y$ is used for all $X$ in that range. This means that $\mathbb{E}(Y|X)$ is the same for all $X$ in that interval, i.e. is just a constant function of $X$. So long as it is a different constant function of $X$ over a different interval of $X$, then the conditional mean must be a non-linear function of $X$ overall.

I went for the easiest case of splitting the distribution of $X$ up into two parts with probability of $\frac{1}{2}$ each. After that, it was clear that the half-normal distribution would be a good choice for the conditional distribution of $Y$ - it's just a matter of gluing two sides back together to get the entire normal distribution as the marginal. If I'd wanted to something fancier, I could have cut $X$ up into many such chunks, possibly irregular, and selected appropriate truncated normals for the conditional distributions of $Y$ over each chunk. The question asks for the underlying intuition, and this is the thought process that resuilted in my solution, but if you want to generalise more widely you are best to think about the copula.

  • 5
    $\begingroup$ This is Example 3 in Moderator cardinal's answer (attributed to an answer of mine though I lay no claim to being the first one to come up with the idea) to the duplicate question pointed out by Moderator whuber. Also, may I suggest that you amend your last paragraph to clarify that it is $E[Y\mid X]$ that has value $\pm\sqrt{\frac{2}{\pi}}$ according as $X$ is positive or negative and so is a nonlinear function of $X$? $\endgroup$ – Dilip Sarwate Dec 4 '14 at 14:46
  • 2
    $\begingroup$ @dilip Yes I noticed it matched Example 3 when whuber marked the duplicate. I have tried to clarify as you suggested. Since the question asked for "intuition" I have attempted to explain the thought process, though I'm unsure how successful this has been! $\endgroup$ – Silverfish Dec 4 '14 at 15:24
  • 3
    $\begingroup$ It is interesting to compare $E[Y\mid X]$, the optimum (not necessarily linear) minimum-mean-square-error estimator for $Y$ given $X$ with the linear minimum-mean-square-error estimator for $Y$ given $X$. For jointly normal random variables, the two estimators are the same, while in this instance they are not. $\endgroup$ – Dilip Sarwate Dec 4 '14 at 16:13

Not the answer you're looking for? Browse other questions tagged or ask your own question.