1. How is the constant term in a linear least squares regression different from the error term?
  2. Why does the error term have a normal distribution with mean "0" and sd $\sigma$? I mean I can see that the error term has a normal distribution since the values of the dependent variable are spread out unlike logistic regression but I couldn't understand why they must necessarily have a zero mean.

1 Answer 1

  1. The constant is in most models meaningless. However, it is important to include it because it ensures that your residuals, i.e. your estimates of the error term, have mean zero. So, the difference to the error term is, that it does not vary whereas the error term does. Moreover, after estimating your model, you include the constant when making predictions, however, there is no longer an error term included.

  2. When deterministic regressors are assumed and the model is linear in its parameters, one of the fundamental theorems in OLS estimation is the Gauss-Markov theorem. It states that under some conditions for the error term, OLS provides the best linear unbiased estimator for a model and its parameters. Hence, OLS is said to be BLUE. These conditions are:

a) $E[u] = 0$

b) $V[u_i] = \sigma^2 < \infty \ \forall \ i \implies$ constant variance, i.e $u$ does not depend on the predictors

c) $E[u_i u_j]=0 \ \forall \ i \neq j \implies$ no correlation.

$E[u] = 0$ means in this context, the error varies evenly around your predictions.

Please note: If the regressors are stochastic, the Gauss-Markov theorem can be restated in terms of an error that is conditioned on the predictors. Then, in the case of strict exogeneity, the Gauss-Markov theorem can be also applied.

Many thanks to the comments of Alecos Papadopoulos and Mico.

  • 2
    $\begingroup$ You omitted a crucial condition of the GM Theorem: that the regressors are nonrandom. This is a much stronger condition than the assumption that the error terms are regressors are mutually independent. $\endgroup$
    – Mico
    Commented Jan 11, 2015 at 18:37
  • $\begingroup$ Hi Mico, thanks for your comment. I stated only the assumptions regarding the error term because I the questions was only concerned about the error term. But of course, you are right. $\endgroup$
    – random_guy
    Commented Jan 11, 2015 at 18:56
  • 1
    $\begingroup$ Gauss Markov holds re-stated in terms of the conditional variance when the regressors are random variables. $\endgroup$ Commented Jan 12, 2015 at 22:03
  • $\begingroup$ Good catch! I added it to make the answer more complete. $\endgroup$
    – random_guy
    Commented Jan 12, 2015 at 22:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.