Skip to main content

The residuals of a model are the actual values minus the predicted values. Many statistical models make assumptions about the error, which is estimated by the residuals.

In the context of regression, the $i^{th}$ residual is defined as:

$$\epsilon_i = y_i - \hat{y}_i$$

Where $y_i$ is the actual value of the $i^{th}$ observation and $\hat{y}_i$ is its estimated or fitted value.

The sum of residuals in a least squares regression is 0, i.e. $\sum_i \epsilon_i = 0$. And the goal of least squares is to find the $\beta$ which minimizes the sum of squared residuals (SSR), i.e.

$$\hat{\beta}_{OLS} = \underset{\beta} {\text{argmin}} \sum_i \epsilon_i^2 = \underset{\beta} {\text{argmin}} \sum_i ( y_i - \sum_j x_{ij}\beta_j)^2$$