# Examine whether simple linear regression of y on x is the same as x on y

I want to determine whether or not I get the same regression results when doing regression of $$x$$ on $$y$$ and of $$y$$ on $$x$$.

Using R's built in lm function I get the following results.

##
## Call:
## lm(formula = y ~ x, data = df1)
##
## Residuals:
## Min 1Q Median 3Q Max
## -1.92127 -0.45577 -0.04136 0.70941 1.83882
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 3.0001 1.1247 2.667 0.02573 *
## x 0.5001 0.1179 4.241 0.00217


And

##
## Call:
## lm(formula = x ~ y, data = df1)
##
## Residuals:
## Min 1Q Median 3Q Max
## -2.6522 -1.5117 -0.2657 1.2341 3.8946
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) -0.9975 2.4344 -0.410 0.69156
## y 1.3328 0.3142 4.241 0.00217


I figured that if the regression lines are the same then

$$y_1 = \alpha + \beta x_1 \Longleftrightarrow x_1 = \frac{y_1- \alpha}{\beta}$$

from lm(y ~ x, data = df1) and

$$x_2 = \alpha_2 + \beta_2 y_2$$

from lm(x ~ y, data = df1) should match up. (Is this correct?)

In my case that would give us (for $$y = 1$$)

\begin{align*}x_1 = \frac{y_1- \alpha}{\beta} = \frac{1 - 3.0001}{0.5001} \approx -3.9994 \\ x_2 = \alpha_2 + \beta_2 y_2 = -0.9975 + 1.3328y = 0.3353 \end{align*}

So $$x_1 \neq x_2$$ and thus there is a difference between linear regression of $$y$$ on $$x$$ and that of $$x$$ on $$y$$.

Is this correct?

• You are forgetting the error term in your equations. There are other issues as well but that should be the start. Please also see this. Commented Oct 30, 2020 at 17:33

In the case of a simple linear regression:

$$y = \alpha + \beta x + \epsilon$$

Beta can be estimated via $$\beta = \frac{\text{Cov}(x,y)}{\text{Var}(x)}$$. And so if we flip x and y, the covariance stays the same, it is only the denominator part for the variance that changes. So from there I imagine you can work out when they will (or will not) be equal!

It depends on your loss function. A common way is to minimalize the residual sum of squares (case $$y \sim x$$):

$$\sum_{i=1}^n (y_i - \alpha - \beta x_i)^2 \rightarrow min$$

This is what your function in R does. It takes into consideration only the vertical distance (in the case when $$y$$ is your vertical axis).

By slipping $$x$$ and $$y$$ it will be the originial horizontal distance minimilized (after summation, of course).

So it is not the same, but there exist other methods too. As a loss function you can choose the Euclidean distance of the points from the regression line and minimize the sum of those errors. In this case your solution should work.