5
$\begingroup$

Following is a note from page 1 in: Matrix OLS

Make sure that you are always careful about distinguishing between disturbances ($\epsilon$) that refer to things that cannot be observed and residuals ($e$) that can be observed. It is important to remember that $\epsilon$ $\not =$ $e$.

What is the difference between $\epsilon$ and $e$? Given that $\epsilon$ is considered unobserved, in what sense are we able to use this value for OLS?

$\endgroup$

2 Answers 2

6
$\begingroup$

The disturbance, or error term, represents factors other than x that affect y. Understanding the error term is useful in OLS because it is the subject of many of our important assumptions.

Since we do not observe errors, we resort to looking at residuals, which can give us an idea about the underlying errors. While errors are unobservable, residuals are observable: we can calculate residuals; that is, we can calculate the difference between each of our y values and their corresponding fitted values that lie on the regression line.

As for the aforementioned assumptions, two important ones of the Classical Linear Model Assumptions are (1) Normality of Error Terms and (2) Homoscedasticity.

Assumption (1): We assume that the unobserved factors are normally distributed around the population regression function. This assumption is for the purpose of statistical inference, but it is not crucial: this assumption can be replaced by the assumption of a large sample size.

Assumption (2): We assume that the error $\epsilon$ has the same variance given any value of the explanatory variables. This assumption is critical in OLS. If the model exhibits heteroscedasticity, then our t statistics do not have t distributions, and our f statistics do not have f distributions; and thus, our statistical inference is no longer reliable. This assumption CANNOT be replaced by the assumption of a large sample size. Though, there are methods for dealing with heteroscedasticity.

There are also other assumptions from the Classical Linear Model Assumptions that rely on our understanding the error term, but those are beyond the scope of this post.

One last note: we have assumed use of cross-sectional data.

$\endgroup$
4
$\begingroup$

https://en.wikipedia.org/wiki/Errors_and_residuals

The "disturbance" or "error" is the difference between the population mean and the observed value.

The "residual" is the difference between the sample mean and the observed value.

The sum of the residuals is necessarily zero. The sum of the disturbances is, with probability $1$, not zero.

The disturbances are independent. The residuals cannot be independent since they must sum to zero. (So, for example, if you add up all but one of the residuals and the sum is $+8$, then the remaining residual must be $-8$. It can be predicted based on the other residuals, and so cannot be independent of them.)

In linear regression, not only is the sum of the residuals necessarily zero, but the sum of products of the residuals and corresponding values of any of the predictor variables must also be zero. In such a case, the residual is not the difference from the sample mean, but the difference from the fitted value.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.