In a simple linear regression setting with homoscedastic errors, we have the following population equations depending on regressor and regressand. I put a subscript to the error term in order to emphasize that they are different:
(1)$$y = \frac{cov(x,y)}{var(x)}x + \epsilon_x$$ (2)$$x = \frac{cov(x,y)}{var(y)}y + \epsilon_y$$
Now for the first equation I derive the variance of the error term (assuming $x$ and $\epsilon_x$ are uncorrelated):
$$var[y] = var[\frac{cox(x,y)}{var(x)}x + \epsilon_x] = \frac{cox(x,y)}{var(x)}var(x) + var[\epsilon_x] = cov(x,y) + var[\epsilon_x]$$ $$\implies var[\epsilon_x] = var(y) - cov(x,y)$$
similarly for the second one:
$$var[\epsilon_y] = var(x) - cov(x,y)$$
So far nothing interesting. What I was hoping is that if I reorganize the first regression
$$x = \frac{var(x)}{cov(x,y)}y - \frac{var(x)}{cov(x,y)}\epsilon_x$$
the variance of the new error term would match the variance of the second regression. But it doesn't.
$$var[-\frac{var(x)}{cov(x,y)}\epsilon_x] \neq var[\epsilon_y]$$
It is puzzling that we don't get the same minimum error variance without actually running regression (2). I presume this is due to the fact that $y$ and $\epsilon_x$ are correlated.
Is my thinking correct?
Is it true that:
$$var[-\frac{var(x)}{cov(x,y)}\epsilon_x] \geq var[\epsilon_y]$$
Is it possible to figure out the minimum error variance regression without actually running both regressions if we assume that we will use 1 unit of x (as equations can be scaled arbitrarily) in both cases without scaling x or y (i.e. to unit variance)?
