We have an equation with $\alpha =0$ therefore the simple linear regression is: $$ Y_i=\beta X_i+u_i$$ and the question was. Derive the OLS $\beta $ and provide comments on its variance. In this situation should I have derived the OLS the same way as for a regression with intercept? to obtain $$ \hat\beta =\frac{Cov(XY)}{Var(X)} $$ or is it something different that I should have done?

  • $\begingroup$ What do you mean by "the normal way"? $\endgroup$
    – user158565
    May 8 '17 at 17:56
  • $\begingroup$ I mean the normal derivation of ols for a simple linear regression which has an intercept $\endgroup$ May 9 '17 at 6:30

That is not correct if there is no intercept. Without an intercept the OLS estimate would be:

$$ \beta = \frac{\operatorname{E}[XY]}{\operatorname{E}{[X^2]}}$$

In the case of a finite sample, your estimate would be:

$$ \hat{\beta} = \frac{\sum_{i=1}^n x_i y_i }{\sum_{i=1}^n x_i^2} $$

The general case (skip over this if you don't know matrix algebra yet)

If you know matrix algebra, all these are special cases. Minimizing the sum of squares can be written as:

$$ \text{minimize (over $\mathbf{b}$) } \quad\left( \mathbf{y} - X \mathbf{b}\right)'\left( \mathbf{y} - X \mathbf{b}\right) $$

Which has the solution:

$$ \hat{\mathbf{b}} = (X'X)^{-1} X'\mathbf{y}$$

The algebra behind that can be found (among numerous places) on my answer here.

  • $\begingroup$ So beta is the same for a regression which has an intercept and for a regression which does not have an intercept, right? $\endgroup$ May 9 '17 at 6:32
  • $\begingroup$ Sorry I am looking at your equation and it seems like it is the same as the one I wrote in my question: $ \hat\beta = \frac{\sum (x_i - \bar x)(y_i - \bar y)}{\sum (x_i -\bar x)^2 } $ $\endgroup$ May 9 '17 at 9:13
  • $\begingroup$ @Alex They are not the same unless $\bar{x} = 0$ and $\bar{y} = 0$. A regression line without an intercept term is forced to go through the origin. $\endgroup$ May 9 '17 at 12:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.