# $\epsilon$ vs residual

In section 3.2.3 of Elements of statistical learning, the authors wrote in equation 3.23

$$Y = X\beta + \epsilon$$

but didn't give a name for $$\epsilon$$. Then it states that the "residuals" are $$r_i = y_i - x_i \hat{\beta}$$

What is the formal terminology for $$\epsilon$$ and how does it relate to the residuals, $$r_i$$?

My understanding is that $$\epsilon$$ represents the deviation between $$Y$$ and $$X\beta$$. and $$r_i$$ represents the deviation between $$Y$$ and $$X \hat{\beta} = \hat{Y}$$. So is $$r$$ an approximation of $$\epsilon$$?

• The notation $\epsilon$ (epsilon) is often tied to the name "error term" and alliteration is one element there, as epsilon was (and is) the Greek letter $e$. Although historical, the idea of an error term or disturbance (for which $u$ is also common notation) is broadly a nuisance insofar as it leads to an expectation among learners (all of us, some of the tme) that other quite different models can be thought of as including error terms, yet often they can't (or can't easily). – Nick Cox May 21 at 6:43

So in the case where the model parameters are unbiased and an approximation of the "real" underlying distribution, r will be an approximation of $$\epsilon$$, and furthermore the distribution of r's for a given vector of $$X$$ will an approximation to the real error distribution.
• @Tim Ah I see. But I was specifically wondering, when we write $Y = X\beta + \epsilon$, we are saying that $f(\beta) = X\beta$ is the true model? And if so, in this case we are saying the true model is linear? If so, I am confused how we can say the "true model" is linear, if the true model isn't observed. – user5965026 May 21 at 18:10
• Sorry above should say $f(X)$ instead of $f(\beta)$ – user5965026 May 21 at 18:43