# Where does this formula for the OLS estimate as a function of the true parameters come from?

Suppose that in the population: $$y = \alpha + \beta_1 x_1 + \epsilon$$

We now estimate the model: $$\hat{y} = \hat{\alpha} + \hat{\beta_1} x_1$$

I have seen the following formula which writes the OLS estimate as a function of the true parameter:

$$\hat{\beta}_\text{OLS} = \beta + \text{corr}(x_1, \epsilon) \dfrac{\sigma_\epsilon} {\sigma_{x_1}}$$

Where does this formula comes from?

Do you now nay bibliographic references where I can find a proof?

• I think that is just computing the residuals you would get from a linear regression model with the population parameter values and then regressing those residuals again on x. The overall estimated beta would be the beta your would get from that, $\mathrm{corr}(x_1,\epsilon)\frac{\sigma_\epsilon}{\sigma_{x_1}}$, plus the population value, $\beta$? Commented Jul 26, 2021 at 13:59

$$\hat \beta_{OLS} = \frac{\sum_{i=1}^n(x_{i1}-\bar x_1)(y_{i}-\bar y)}{\sum_{i=1}^n(x_{i1}-\bar x_1)^2}.$$
Plugging in $$y_i = \alpha + \beta x_{i1} + \varepsilon_i$$ and $$\bar y = \alpha + \beta \bar x_{1} + \bar \varepsilon$$, we get: $$\hat \beta_{OLS} = \frac{\beta \sum_{i=1}^n(x_{i1}-\bar x_1)^2 + \sum_{i=1}^n(x_{i1}-\bar x_1)(\varepsilon_{i}-\bar \varepsilon)}{\sum_{i=1}^n(x_{i1}-\bar x_1)^2} = \beta + \frac{\sum_{i=1}^n(x_{i1}-\bar x_1)(\varepsilon_{i}-\bar \varepsilon)}{\sum_{i=1}^n(x_{i1}-\bar x_1)^2}$$
Imagine multiplying both numerator and denominator with $$1/(n-1)$$, then: $$\hat \beta_{OLS} = \beta + \frac{\widehat{cov}(x_1, \varepsilon)}{\hat \sigma_{x_1}^2} = \beta + \widehat{corr}(x_1, \varepsilon)\frac{\hat\sigma_{\varepsilon}}{\hat\sigma_{x_1}},$$ where $$\widehat{cov}$$ represents the sample covariance, $$\hat \sigma^2$$ sample variance, $$\hat \sigma$$ sample standard deviation and $$\widehat{corr}$$ sample correlation.
• Thank you very much. Am I correct in saying that $x_1$ and $y$ in your formulae are the vectors containing data from the entire population, and if you instead sample from the population you introduce another "error" in $\hat{\beta}_\text{OLS}$ which is the sampling error? Commented Jul 26, 2021 at 15:19
• I think your intuition is close to the truth. When the ideal conditions of the OLS are satisfied, the population moment $cov(x_1,\varepsilon) = 0$. As we have only a sample at hand, we can only estimate the covariance, i.e. calculate $\widehat{cov}(x_1,\varepsilon)$ for example using a sample covariance formula. The discrepancy between them would add to the sampling error. Note that high variance in the regressor $x_1$ lowers the error. Commented Jul 27, 2021 at 17:24