# In OLS, why does the error term equal $\mathbf{y} - \hat{\mathbf{y}}$?

In OLS, why does the error term equal $$\mathbf{y} - \hat{\mathbf{y}}$$? I would have assumed that

\begin{aligned} \mathbf{y} - \hat{\mathbf{y}} &= \mathbf{X}\boldsymbol{\beta} + \boldsymbol{\varepsilon} - \mathbf{X}\boldsymbol{\hat{\beta}} \\ &= \mathbf{X}(\boldsymbol{\beta} - \boldsymbol{\hat{\beta}}) + \boldsymbol{\varepsilon} \end{aligned}

The only thing I can think of is that we assume $$\boldsymbol{\beta} = \hat{\boldsymbol{\beta}}$$, and that all the error is captured by $$\boldsymbol{\varepsilon}$$. I can't find this assumption in a textbook, though.

• What your derivation shows is that if your estimator $\hat\beta$ were to be exactly equal to the true parameter $\beta$ (whether that is the case is unknown in practice as we do statistics precisely because we do not know $\beta$), then residuals $\hat\epsilon$ and errors $\epsilon$ would be identical. Jun 30, 2021 at 6:37
• That makes sense. Thanks.
– jds
Jun 30, 2021 at 13:24

The residual (you don’t mean “error”) is by how much your estimate misses the observed value.

You observe $$y$$.

You estimate $$\hat y$$.

You missed by $$y-\hat y$$.

The “error” term is unobserved but estimated by the residuals.

• I see. So $\hat{\varepsilon}$ is the residual, which is an estimate of the error $\varepsilon$. We simply define it to be $y - \hat{y}$? I was a little surprised, because I expected us to disambiguate between the error in the estimate of $\beta$ and the unavoidable error implicit in $\varepsilon$. But $\hat{\varepsilon}$ captures both.
– jds
Jun 30, 2021 at 13:24