3
$\begingroup$

Suppose that in the population: $$ y = \alpha + \beta_1 x_1 + \epsilon $$

We now estimate the model: $$ \hat{y} = \hat{\alpha} + \hat{\beta_1} x_1 $$

I have seen the following formula which writes the OLS estimate as a function of the true parameter:

$$ \hat{\beta}_\text{OLS} = \beta + \text{corr}(x_1, \epsilon) \dfrac{\sigma_\epsilon} {\sigma_{x_1}}$$

Where does this formula comes from?

Do you now nay bibliographic references where I can find a proof?

$\endgroup$
1
  • $\begingroup$ I think that is just computing the residuals you would get from a linear regression model with the population parameter values and then regressing those residuals again on x. The overall estimated beta would be the beta your would get from that, $\mathrm{corr}(x_1,\epsilon)\frac{\sigma_\epsilon}{\sigma_{x_1}}$, plus the population value, $\beta$? $\endgroup$ Commented Jul 26, 2021 at 13:59

1 Answer 1

1
$\begingroup$

Starting from the standard result for a bivariate regression model, we have:

$$ \hat \beta_{OLS} = \frac{\sum_{i=1}^n(x_{i1}-\bar x_1)(y_{i}-\bar y)}{\sum_{i=1}^n(x_{i1}-\bar x_1)^2}. $$

Plugging in $y_i = \alpha + \beta x_{i1} + \varepsilon_i$ and $\bar y = \alpha + \beta \bar x_{1} + \bar \varepsilon$, we get: $$ \hat \beta_{OLS} = \frac{\beta \sum_{i=1}^n(x_{i1}-\bar x_1)^2 + \sum_{i=1}^n(x_{i1}-\bar x_1)(\varepsilon_{i}-\bar \varepsilon)}{\sum_{i=1}^n(x_{i1}-\bar x_1)^2} = \beta + \frac{\sum_{i=1}^n(x_{i1}-\bar x_1)(\varepsilon_{i}-\bar \varepsilon)}{\sum_{i=1}^n(x_{i1}-\bar x_1)^2} $$

Imagine multiplying both numerator and denominator with $1/(n-1)$, then: $$ \hat \beta_{OLS} = \beta + \frac{\widehat{cov}(x_1, \varepsilon)}{\hat \sigma_{x_1}^2} = \beta + \widehat{corr}(x_1, \varepsilon)\frac{\hat\sigma_{\varepsilon}}{\hat\sigma_{x_1}}, $$ where $\widehat{cov}$ represents the sample covariance, $\hat \sigma^2$ sample variance, $\hat \sigma$ sample standard deviation and $\widehat{corr}$ sample correlation.

$\endgroup$
2
  • $\begingroup$ Thank you very much. Am I correct in saying that $x_1$ and $y$ in your formulae are the vectors containing data from the entire population, and if you instead sample from the population you introduce another "error" in $\hat{\beta}_\text{OLS}$ which is the sampling error? $\endgroup$ Commented Jul 26, 2021 at 15:19
  • $\begingroup$ I think your intuition is close to the truth. When the ideal conditions of the OLS are satisfied, the population moment $cov(x_1,\varepsilon) = 0$. As we have only a sample at hand, we can only estimate the covariance, i.e. calculate $\widehat{cov}(x_1,\varepsilon)$ for example using a sample covariance formula. The discrepancy between them would add to the sampling error. Note that high variance in the regressor $x_1$ lowers the error. $\endgroup$
    – FarFarAway
    Commented Jul 27, 2021 at 17:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.