From Applied Linear Regression (Weisberg S., 2014):

The errors $e$ are unobservable random variables, with $\text{E}(e|X) = 0$ and $\text{Var}(e|X) = σ^2 I$. The residuals $\hat e$ are computed quantities that can be graphed or otherwise studied.

What does "unobservable random variable" mean in the above quote? Is it incorrect to say that the errors are random variables and the residuals particular observations of said random variables?

  • $\begingroup$ Unfortunately, we don't know the true relationship between the inputs and outputs, which makes sence since the goal of Linear Regression is to estimate the functional relationship between them. Since we don't have access to that relationship, we can't know what the errors (i.e. deviations from that relationship) are. $\endgroup$ – David Kozak Feb 1 '17 at 0:34
  • 1
    $\begingroup$ The residuals are also random variables but, no, they are not realizations of the errors. The residuals are estimates of the errors and, as the the sample size increases, they converge to the true errors (because the coefficient estimates converge to the true values) $\endgroup$ – gammer Feb 1 '17 at 3:41

You can measure $X$ and $y$ directly, but you do not have any direct measures of $e$. You can estimate $e$ from the data using a regression model, but that is not the same as measuring $e$ directly. Thefore, errors are unobservable.

As @gammer put it in a comment, the residuals are estimates of errors. They converge to the true values as the estimate of the regression coefficient converges to its respective true value when sample size grows.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.