2
$\begingroup$

for a simple linear model:

$y= \alpha+\beta x+e$

$\hat \beta= \frac{cov(x,y)}{var(x)} = $ second element of $(X'X)^{-1}X'y $

where $X = [1, x]$

However, if the true relation is

$y= \beta x+e$ with $cov(x,e)=0$

the cov formula still works as

$\frac{cov(x,y)}{var(x)} = \frac{cov(x,(\beta x+e))}{var(x)} = \frac{\beta*cov(x,x)+cov(x,e)}{var(x)} = \beta $

But I am trying to understand why the least squares doesnt work anymore:

$(X'X)^{-1}X'y = (X'X)^{-1}X'(X\beta+e) = \beta + (X'X)^{-1}X'e = \ \beta$

But now for true model $(X'X)^{-1}X'y = \frac{\sum(xy)}{\sum(x^2)} \not= \frac{cov(x,y)}{var(x)} $ for model with a constant.

Can someone point out if I might be mistaken in thinking that regular OLS with constant is still unbiased and consistent? How can both of these be unbiased as they are different.

In other words, are both $\frac{\sum(xy)}{\sum(x^2)} $ and $ \frac{cov(x,y)}{var(x)} $ unbiased/consistent when true $\alpha=0$ and how (assuming $E(x), E(y) \not=0$)?

$\endgroup$
4
$\begingroup$

Yes, both are unbiased when $\alpha=0$. If the true model does not contain a constant, the regression that does fit one contains a superfluous regressor. This is not good for efficiency, but has no impact on unbiasedness.

For the opposite case in which the true model does contain a nonzero intercept but a regression is fit without one, see for example this post.

You do not distinguish very clearly between sample and population (co)variances, but note that, e.g., the population covariance is generally defined as $$ E[(X-E(X))(Y-E(Y))] $$ When $E(X)=E(Y)=0$, this evidently simplifies to $E[XY]$. So, in that sense, your final display is true in the case of a true model without constant.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ @Christopher_Hanck , yes exactly what I'm trying to understand. How are they both unbiased in case of nonzero means. E(X) or E(y) need not be zero in case of non constant, right? $\endgroup$ – dayum May 16 '18 at 22:38
  • 1
    $\begingroup$ See e.g. stats.stackexchange.com/questions/120731/… $\endgroup$ – Christoph Hanck May 17 '18 at 8:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.