Examine whether simple linear regression of y on x is the same as x on y I want to determine whether or not I get the same regression results when doing regression of $x$ on $y$ and of $y$ on $x$.
Using R's built in lm function I get the following results.
##
## Call:
## lm(formula = y ~ x, data = df1)
##
## Residuals:
## Min 1Q Median 3Q Max
## -1.92127 -0.45577 -0.04136 0.70941 1.83882
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 3.0001 1.1247 2.667 0.02573 *
## x 0.5001 0.1179 4.241 0.00217

And
##
## Call:
## lm(formula = x ~ y, data = df1)
##
## Residuals:
## Min 1Q Median 3Q Max
## -2.6522 -1.5117 -0.2657 1.2341 3.8946
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) -0.9975 2.4344 -0.410 0.69156
## y 1.3328 0.3142 4.241 0.00217

I figured that if the regression lines are the same then
$$y_1 = \alpha + \beta x_1 \Longleftrightarrow x_1 = \frac{y_1- \alpha}{\beta}$$
from lm(y ~ x, data = df1) and
$$x_2 = \alpha_2 + \beta_2 y_2$$
from lm(x ~ y, data = df1) should match up. (Is this correct?)
In my case that would give us (for $y = 1$)
$$\begin{align*}x_1 = \frac{y_1- \alpha}{\beta} = \frac{1 - 3.0001}{0.5001} \approx -3.9994 \\ x_2 = \alpha_2 + \beta_2 y_2 = -0.9975 + 1.3328y = 0.3353 \end{align*}$$
So $x_1 \neq x_2$ and thus there is a difference between linear regression of $y$ on $x$ and that of $x$ on $y$.
Is this correct?
Thanks in advance.
 A: In the case of a simple linear regression:
$$y = \alpha + \beta x + \epsilon$$
Beta can be estimated via $\beta = \frac{\text{Cov}(x,y)}{\text{Var}(x)}$. And so if we flip x and y, the covariance stays the same, it is only the denominator part for the variance that changes. So from there I imagine you can work out when they will (or will not) be equal!
A: It depends on your loss function. A common way is to minimalize the residual sum of squares (case $y \sim x$):
$$ \sum_{i=1}^n (y_i - \alpha - \beta x_i)^2 \rightarrow min$$
This is what your function in R does. It takes into consideration only the vertical distance (in the case when $y$ is your vertical axis).
By slipping $x$ and $y$ it will be the originial horizontal distance minimilized (after summation, of course).
So it is not the same, but there exist other methods too. As a loss function you can choose the Euclidean distance of the points from the regression line and minimize the sum of those errors. In this case your solution should work.
