- How is the constant term in a linear least squares regression different from the error term?
- Why does the error term have a normal distribution with mean "0" and sd $\sigma$? I mean I can see that the error term has a normal distribution since the values of the dependent variable are spread out unlike logistic regression but I couldn't understand why they must necessarily have a zero mean.
The constant is in most models meaningless. However, it is important to include it because it ensures that your residuals, i.e. your estimates of the error term, have mean zero. So, the difference to the error term is, that it does not vary whereas the error term does. Moreover, after estimating your model, you include the constant when making predictions, however, there is no longer an error term included.
When deterministic regressors are assumed and the model is linear in its parameters, one of the fundamental theorems in OLS estimation is the Gauss-Markov theorem. It states that under some conditions for the error term, OLS provides the best linear unbiased estimator for a model and its parameters. Hence, OLS is said to be BLUE. These conditions are:
a) $E[u] = 0$
b) $V[u_i] = \sigma^2 < \infty \ \forall \ i \implies$ constant variance, i.e $u$ does not depend on the predictors
c) $E[u_i u_j]=0 \ \forall \ i \neq j \implies$ no correlation.
$E[u] = 0$ means in this context, the error varies evenly around your predictions.
Please note: If the regressors are stochastic, the Gauss-Markov theorem can be restated in terms of an error that is conditioned on the predictors. Then, in the case of strict exogeneity, the Gauss-Markov theorem can be also applied.
Many thanks to the comments of Alecos Papadopoulos and Mico.