# For a linear regression of Y on X, when is the regression of X on Y linear? When is it non-linear?

Suppose we have two random variables $$X_1$$ and $$X_2$$ that both have finite expectations, that may or may not be correlated, and that are known to be linearly related (i.e., I mean their sum forms a straight line of constant slope) according to the equation, $$Y = aX_1 + bX_2 .$$ We know, by construction, that $$E[Y|X_1=x_1]$$ is is a straight line with constant slope. Then, when is $$E[X_1|Y=y]$$ also linear in the same sense?

Based on simulations, I believe that $$E[X_1|Y=y]$$ will be a straight line with constant slope if $$X_1$$ and $$X_2$$ are i.i.d., but I can't find a proof. It is certainly linear when $$X_1$$ and $$X_2$$ are both independent Normal.

One simple example of when $$E[X_1|Y=y]$$ is not linear (does not have constant slope) is when $$X_1\sim Uniform[-c_1,c_1]$$ and $$X_2\sim Uniform[-c_2,c_2]$$ when $$c_1\neq c_2$$. In this case, $$E[X_1|Y=y]$$ has a zigzag shape, increasing initially for small y then flattening out at zero for a while then increasing again.

So, is there a proof that $$E[X_1|Y=y]$$ is a straight line with constant slope for i.i.d. $$X_1$$ and $$X_2$$, and are there instances when $$E[X_1|Y=y]$$ is linear but $$X_1$$ and $$X_2$$ are not i.i.d.?

Any help is much appreciated!

EDIT: Here are examples below demonstrating a case when $$E[X_1|Y=y]$$ is not linear (i.e., does not form a straight line with constant slope despite $$E[Y|X_1]$$ being linear), specifically with $$X_1 \sim Uniform[-2,2]$$ and $$X_2 \sim Uniform[-3,3]$$ and coefficients a = b = 1. The first two plots are simulated data, where the black line is a moving average. The third plot is the actual numerical solution found by computing the conditional pdf and integrating.

• The question is clear, but your terminology might confuse readers. Your initial definition of "linearly related" is without content: all pairs of random variables $(X_1,X_2)$ enjoy such a relation. See stats.stackexchange.com/questions/257779 for a somewhat related question. For insight into all three of your questions, think of $X_1$ as an "independent variable," $bX_2$ as an "error term," and draw pictures.
– whuber
Jun 24 '21 at 13:57
• @whuber I am indeed thinking of bX2 as an error term. I am trying to figure out how to post images so that I can post images of the numerical and simulation output to demonstrate what I mean . . . Jun 24 '21 at 14:18
• @whuber ok, figures added. How should I phrase the linearity part to make it more accurate? Jun 24 '21 at 14:36
• The linear relation to which you refer is between $X_1$ and $Y,$ not between $X_1$ and $X_2.$ If you were to plot the curve $y=ax_1$ on $(x_1,y)$ axes, then adding $bX_2$ will alter the heights of that graph randomly according to the distribution of $X_2.$ Since you allow $X_1$ and $X_2$ to be correlated (you don't even require the mean of $X_2$ conditional on $X_1$ be zero), those alterations may follow literally any path--even a deterministic one. Drawing a few such plots will reveal why the regression of $X_1$ against $Y$ can be almost arbitrarily complicated.
– whuber
Jun 24 '21 at 15:32
• @whuber yes, I agree that the regression of $X_1$ on $Y$ can be almost arbitrarily complicated. Thus my question: when is $E[X_1|Y=y]$ a straight line? It is a straight line when $X_1$ and $X_2$ are iid Normal. What other conditions? Is iid $X_1$ and $X_2$ also sufficient? Jun 24 '21 at 15:42

Suppose $$X_1$$ and $$X_2$$ are iid and $$Y = X_1+X_2$$, then $$E[X_1|Y=y]$$ must be a straight line with slope 1/2.
Proof: Solving for $$X_1$$ and taking the conditional expectation gives $$E[X_1|Y=y]=y-E[X_2|Y=y]$$ If $$X_1$$ and $$X_2$$ are iid, then $$E[X_1|Y=y]=E[X_2|Y=y]=y/2$$, which is a straight line with constant slope equal to 1/2.
• +1. The relevant concept is exchangeability, implied by the iid assumption. Because of that and linearity of conditional expectation, $$y = E[Y\mid Y=y] = E[X_1+X_2\mid X_1+X_2=y] = E[X_1\mid X_1+X_2=y] + E[X_2\mid X_1+X_2=y].$$ Since the two terms are equal (by exchangeability), each must equal $y/2,$ QED.