# We know beta for univariate OLS; what can we say about beta after switching the independent and dependent variables?

I was asked this in an interview, and I'm curious if (1) my reasoning was correct and (2) if I could have been more precise in my answer. Consider standard OLS with a single predictor:

$$\mathbf{y} = \mathbf{x} \beta_1 + \boldsymbol{\varepsilon}_1. \tag{1}$$

Now let's say that $$\hat{\beta}_1 = 10$$. Now imagine we switched the independent and dependent variables, i.e.

$$\mathbf{x} = \mathbf{y} \beta_2 + \boldsymbol{\varepsilon}_2. \tag{2}$$

Can we say anything about the range of values that $$\hat{\beta}_2$$ can take? Here is my attempt. The normal equations for each model are:

\begin{aligned} \hat{\beta}_1 &= (\mathbf{x}^{\top} \mathbf{x})^{-1} \mathbf{x}^{\top} \mathbf{y}, \\ \hat{\beta}_2 &= (\mathbf{y}^{\top} \mathbf{y})^{-1} \mathbf{y}^{\top} \mathbf{x}. \end{aligned} \tag{3}

Since $$\mathbf{x}^{\top} \mathbf{y} = \mathbf{y}^{\top} \mathbf{x}$$, then

\begin{aligned} \hat{\beta}_1 &= 10 \\ &\Downarrow \\ (\mathbf{x}^{\top} \mathbf{x}) 10 &= \mathbf{x}^{\top} \mathbf{y} = \mathbf{y}^{\top} \mathbf{x}. \end{aligned} \tag{4}

Thus, we can write $$\hat{\beta}_2$$ as

\begin{aligned} \hat{\beta}_2 &= (\mathbf{y}^{\top} \mathbf{y})^{-1} (\mathbf{x}^{\top} \mathbf{x}) 10 \\ &= \frac{10 \mathbf{x}^{\top} \mathbf{x}}{\mathbf{y}^{\top} \mathbf{y}}. \end{aligned} \tag{5}

Since the dot products are both positive, clearly $$\hat{\beta}_2 \in [0, \infty)$$, and I think we can even say $$\hat{\beta}_2 \in (0, \infty)$$ if $$\mathbf{x}$$ is a non-zero predictor. In other words, $$\hat{\beta}_2$$ must be non-negative. Intuitively, this makes sense. If $$\hat{\beta}_1$$ is positive, then the relationship between the independent and dependent variables shouldn't change when we switch the regression.

Is this reasoning correct, and can I get a tighter upper bound?

• – whuber
Commented Jul 28, 2021 at 12:13

## 1 Answer

Another way to write this in the univariate case that is a little simpler is that $$\hat{\beta}_1 = \frac{\text{Cov(x, y)}}{\text{Var}(x)}$$. Similarly, $$\hat{\beta}_2 = \frac{\text{Cov(x, y)}}{\text{Var}(y)}$$. So we know (equivalent to what you've written) $$\hat{\beta}_2 = \hat{\beta}_1 \frac{\text{Var}(x)}{\text{Var}(y)}$$. It's true that because the variances are positive, $$\hat{\beta}_1$$ and $$\hat{\beta}_2$$ have the same sign, but because the variances could be absolutely any positive numbers, there is no upper bound you can place on $$\hat{\beta}_2$$.