2
$\begingroup$

Suppose we have two random variables $X_1$ and $X_2$ that both have finite expectations, that may or may not be correlated, and that are known to be linearly related (i.e., I mean their sum forms a straight line of constant slope) according to the equation, $$ Y = aX_1 + bX_2 . $$ We know, by construction, that $E[Y|X_1=x_1]$ is is a straight line with constant slope. Then, when is $E[X_1|Y=y]$ also linear in the same sense?

Based on simulations, I believe that $E[X_1|Y=y]$ will be a straight line with constant slope if $X_1$ and $X_2$ are i.i.d., but I can't find a proof. It is certainly linear when $X_1$ and $X_2$ are both independent Normal.

One simple example of when $E[X_1|Y=y]$ is not linear (does not have constant slope) is when $X_1\sim Uniform[-c_1,c_1]$ and $X_2\sim Uniform[-c_2,c_2]$ when $c_1\neq c_2$. In this case, $E[X_1|Y=y]$ has a zigzag shape, increasing initially for small y then flattening out at zero for a while then increasing again.

So, is there a proof that $E[X_1|Y=y]$ is a straight line with constant slope for i.i.d. $X_1$ and $X_2$, and are there instances when $E[X_1|Y=y]$ is linear but $X_1$ and $X_2$ are not i.i.d.?

Any help is much appreciated!

EDIT: Here are examples below demonstrating a case when $E[X_1|Y=y]$ is not linear (i.e., does not form a straight line with constant slope despite $E[Y|X_1]$ being linear), specifically with $X_1 \sim Uniform[-2,2]$ and $X_2 \sim Uniform[-3,3]$ and coefficients a = b = 1. The first two plots are simulated data, where the black line is a moving average. The third plot is the actual numerical solution found by computing the conditional pdf and integrating. enter image description here enter image description here enter image description here

$\endgroup$
8
  • $\begingroup$ The question is clear, but your terminology might confuse readers. Your initial definition of "linearly related" is without content: all pairs of random variables $(X_1,X_2)$ enjoy such a relation. See stats.stackexchange.com/questions/257779 for a somewhat related question. For insight into all three of your questions, think of $X_1$ as an "independent variable," $bX_2$ as an "error term," and draw pictures. $\endgroup$
    – whuber
    Jun 24 '21 at 13:57
  • $\begingroup$ @whuber I am indeed thinking of bX2 as an error term. I am trying to figure out how to post images so that I can post images of the numerical and simulation output to demonstrate what I mean . . . $\endgroup$
    – user62421
    Jun 24 '21 at 14:18
  • $\begingroup$ @whuber ok, figures added. How should I phrase the linearity part to make it more accurate? $\endgroup$
    – user62421
    Jun 24 '21 at 14:36
  • $\begingroup$ The linear relation to which you refer is between $X_1$ and $Y,$ not between $X_1$ and $X_2.$ If you were to plot the curve $y=ax_1$ on $(x_1,y)$ axes, then adding $bX_2$ will alter the heights of that graph randomly according to the distribution of $X_2.$ Since you allow $X_1$ and $X_2$ to be correlated (you don't even require the mean of $X_2$ conditional on $X_1$ be zero), those alterations may follow literally any path--even a deterministic one. Drawing a few such plots will reveal why the regression of $X_1$ against $Y$ can be almost arbitrarily complicated. $\endgroup$
    – whuber
    Jun 24 '21 at 15:32
  • $\begingroup$ @whuber yes, I agree that the regression of $X_1$ on $Y$ can be almost arbitrarily complicated. Thus my question: when is $E[X_1|Y=y]$ a straight line? It is a straight line when $X_1$ and $X_2$ are iid Normal. What other conditions? Is iid $X_1$ and $X_2$ also sufficient? $\endgroup$
    – user62421
    Jun 24 '21 at 15:42
1
$\begingroup$

So I figured out an answer that I'll leave here in case anyone else has the same question. As a non-math person I could use some help making this more precise, and I would still like to have a citation for this theorem, since this must be a well known result.

Suppose $X_1$ and $X_2$ are iid and $Y = X_1+X_2$, then $E[X_1|Y=y]$ must be a straight line with slope 1/2.

Proof: Solving for $X_1$ and taking the conditional expectation gives $$ E[X_1|Y=y]=y-E[X_2|Y=y] $$ If $X_1$ and $X_2$ are iid, then $E[X_1|Y=y]=E[X_2|Y=y]=y/2$, which is a straight line with constant slope equal to 1/2.

$\endgroup$
2
  • $\begingroup$ +1. The relevant concept is exchangeability, implied by the iid assumption. Because of that and linearity of conditional expectation, $$y = E[Y\mid Y=y] = E[X_1+X_2\mid X_1+X_2=y] = E[X_1\mid X_1+X_2=y] + E[X_2\mid X_1+X_2=y].$$ Since the two terms are equal (by exchangeability), each must equal $y/2,$ QED. $\endgroup$
    – whuber
    Jun 25 '21 at 18:36
  • $\begingroup$ @whuber Thanks! $\endgroup$
    – user62421
    Jun 26 '21 at 2:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.