2
$\begingroup$

In OLS, why does the error term equal $\mathbf{y} - \hat{\mathbf{y}}$? I would have assumed that

$$ \begin{aligned} \mathbf{y} - \hat{\mathbf{y}} &= \mathbf{X}\boldsymbol{\beta} + \boldsymbol{\varepsilon} - \mathbf{X}\boldsymbol{\hat{\beta}} \\ &= \mathbf{X}(\boldsymbol{\beta} - \boldsymbol{\hat{\beta}}) + \boldsymbol{\varepsilon} \end{aligned} $$

The only thing I can think of is that we assume $\boldsymbol{\beta} = \hat{\boldsymbol{\beta}}$, and that all the error is captured by $\boldsymbol{\varepsilon}$. I can't find this assumption in a textbook, though.

$\endgroup$
2
  • 1
    $\begingroup$ What your derivation shows is that if your estimator $\hat\beta$ were to be exactly equal to the true parameter $\beta$ (whether that is the case is unknown in practice as we do statistics precisely because we do not know $\beta$), then residuals $\hat\epsilon$ and errors $\epsilon$ would be identical. $\endgroup$ Jun 30, 2021 at 6:37
  • $\begingroup$ That makes sense. Thanks. $\endgroup$
    – jds
    Jun 30, 2021 at 13:24

1 Answer 1

4
$\begingroup$

The residual (you don’t mean “error”) is by how much your estimate misses the observed value.

You observe $y$.

You estimate $\hat y$.

You missed by $y-\hat y$.

The “error” term is unobserved but estimated by the residuals.

$\endgroup$
1
  • $\begingroup$ I see. So $\hat{\varepsilon}$ is the residual, which is an estimate of the error $\varepsilon$. We simply define it to be $y - \hat{y}$? I was a little surprised, because I expected us to disambiguate between the error in the estimate of $\beta$ and the unavoidable error implicit in $\varepsilon$. But $\hat{\varepsilon}$ captures both. $\endgroup$
    – jds
    Jun 30, 2021 at 13:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.