I regularly see linear regression models written in this notation:

$y = a + \beta X + error$

I've never really pinned down what $error$ actually refers to. In the linear regression plotted below, does $error$ refer to the grey band, therefore the confidence interval for the prediction line?

enter image description here

  • 1
    $\begingroup$ Short answer: no. The error doesn't appear in that image, and is not observed. Your best estimate of the error is the residuals (but sometimes it's a demonstrably poor estimate). $\endgroup$ – Glen_b Feb 6 '14 at 8:12
  • $\begingroup$ By "not observed", do you mean the error is not not observed in the plot or it is not observed in linear regression? $\endgroup$ – luciano Feb 6 '14 at 8:53
  • $\begingroup$ Errors are not observed at all, including in both those senses. If the errors were observed you could calculate the population parameters in simple regression from two observations and their associated errors. The error term is the model for how the data values don't lie on the population line $\endgroup$ – Glen_b Feb 6 '14 at 9:05

The model for simple regression is, as you stated, $Y=\alpha+\beta x+\epsilon$ where $\epsilon$ is there error term. In words, that model states that $Y$ is a random quantity composed of a linear function of a fixed x (that is $\alpha+\beta x$) plus a random error term ($\epsilon$) usually assumed to have mean 0 (and often assumed to be normally distributed).

In simpler terms, under the linear regression model, the error term explains why all the $y$ values do not lie perfectly on the regression line. If the model were simply $Y=\alpha+\beta x$, the points would all lie exactly on the line. The residuals $e_i=y_i-\hat{y_i}$ are estimates of realizations of the error term for individual realizations of $Y$ and $x$.

The grey confidence band in you regression plot captures the uncertainty in the estimated regression line. That is, it represents the uncertainty in the estimates of $\alpha$ and $\beta$. This uncertainty is due to the error term, but is not the same as that error term.


Another common way do describe the error is the following:

Suppose you have some variable $Y$ which you model as a linear function of another variable $X$, that is, you want to see how well the function $f(X):=\alpha+\beta X$ describes $Y$ according to some measure. Then define $\epsilon$ as the difference between $f(X)$ and $Y$. Then $\epsilon$ is all things affecting $Y$ not captured by $f(X)$. To me, this is the most basic and intuitive way to think about the error term.

It's everything in $Y$ not explained by a linear (affine) function of $X$.

For example, if you let $Y=$ average age of citizens in a country, and then you model this as an affine function of how much they exercise and how healthy they eat ($X$="Healthy Lifestyle Index"), then $\epsilon$ is everything affecting how long people live on average except for an affine function of some healthy lifestyle index. This includes effects from smoking, the health care system, happiness, climate, etc.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.