In OLS, why does the error term equal $\mathbf{y} - \hat{\mathbf{y}}$?

Question

In OLS, why does the error term equal $\mathbf{y} - \hat{\mathbf{y}}$? I would have assumed that

$$ \begin{aligned} \mathbf{y} - \hat{\mathbf{y}} &= \mathbf{X}\boldsymbol{\beta} + \boldsymbol{\varepsilon} - \mathbf{X}\boldsymbol{\hat{\beta}} \\ &= \mathbf{X}(\boldsymbol{\beta} - \boldsymbol{\hat{\beta}}) + \boldsymbol{\varepsilon} \end{aligned} $$

The only thing I can think of is that we assume $\boldsymbol{\beta} = \hat{\boldsymbol{\beta}}$, and that all the error is captured by $\boldsymbol{\varepsilon}$. I can't find this assumption in a textbook, though.

Answer 1

The residual (you don’t mean “error”) is by how much your estimate misses the observed value.

You observe $y$.

You estimate $\hat y$.

You missed by $y-\hat y$.

The “error” term is unobserved but estimated by the residuals.