Skip to main content
edited tags
Link
kjetil b halvorsen
  • 82.8k
  • 32
  • 201
  • 663
Source Link
Cagdas Ozgenc
  • 4.5k
  • 4
  • 32
  • 68

Regression Swap X,Y Variance Puzzle

In a simple linear regression setting with homoscedastic errors, we have the following population equations depending on regressor and regressand. I put a subscript to the error term in order to emphasize that they are different:

(1)$$y = \frac{cov(x,y)}{var(x)}x + \epsilon_x$$ (2)$$x = \frac{cov(x,y)}{var(y)}y + \epsilon_y$$

Now for the first equation I derive the variance of the error term (assuming $x$ and $\epsilon_x$ are uncorrelated):

$$var[y] = var[\frac{cox(x,y)}{var(x)}x + \epsilon_x] = \frac{cox(x,y)}{var(x)}var(x) + var[\epsilon_x] = cov(x,y) + var[\epsilon_x]$$ $$\implies var[\epsilon_x] = var(y) - cov(x,y)$$

similarly for the second one:

$$var[\epsilon_y] = var(x) - cov(x,y)$$

So far nothing interesting. What I was hoping is that if I reorganize the first regression

$$x = \frac{var(x)}{cov(x,y)}y - \frac{var(x)}{cov(x,y)}\epsilon_x$$

the variance of the new error term would match the variance of the second regression. But it doesn't.

$$var[-\frac{var(x)}{cov(x,y)}\epsilon_x] \neq var[\epsilon_y]$$

It is puzzling that we don't get the same minimum error variance without actually running regression (2). I presume this is due to the fact that $y$ and $\epsilon_x$ are correlated.

Is my thinking correct?

Is it true that:

$$var[-\frac{var(x)}{cov(x,y)}\epsilon_x] \geq var[\epsilon_y]$$

Is it possible to figure out the minimum error variance regression without actually running both regressions if we assume that we will use 1 unit of x (as equations can be scaled arbitrarily) in both cases without scaling x or y (i.e. to unit variance)?