# Why do residuals in linear regression always sum to zero when an intercept is included?

I'm taking a course on regression models and one of the properties provided for linear regression is that the residuals always sum to zero when an intercept is included.

Can someone provide a good explanation for why this is the case?

• You might like to first ponder the closely related but simpler question of why in a univariate sample, the residuals you obtain by subtracting the sample mean from each value also sum to 0. (Try following the algebra through if you can.) Jan 7, 2016 at 1:50
• As soon as you recognize that "sum to zero" means "orthogonal to one of the explanatory variables" the answer becomes geometrically obvious.
– whuber
May 5, 2017 at 19:57

This follows directly from the normal equations, i.e. the equations that the OLS estimator solves,

$$\mathbf{X}^{\prime} \underbrace{\left( \mathbf{y} - \mathbf{X} \mathbf{b} \right)}_{\mathbf{e}} = 0$$

The vector inside the parentheses is of course the residual vector or the projection of $\mathbf{y}$ onto the orthogonal complement of the column space of $X$, if you like linear algebra. Now including a vector of ones in the $\mathbf{X}$ matrix, which by the way doesn't have to be in the first column as is conventionally done, leads to

$$\mathbf{1}^{\prime} \mathbf{e} = 0 \implies \sum_{i=1}^n e_i = 0$$

In the two-variable problem this is even simpler to see, as minimizing the sum of squared residuals brings us to

$$\sum_{i=1}^n \left(y_i - a - b x_i \right) = 0$$

when we take the derivative with respect to the intercept. From this then we proceed to obtain the familiar estimator

$$a = \bar{y} - b \bar{x}$$

where again we see that the construction of our estimators imposes this condition.

• I'm not sure if I follow the last part where $a = \bar{y} - b \bar{x}$ supposedly implies that the sum of residuals is 0. Could you expand?
– 24n8
Nov 15, 2023 at 2:40

In case you are looking for a rather intuitive explanation.

In some sense, the linear regression model is nothing but a fancy mean. To find the arithmetic mean $$\bar{x}$$ over some values $$x_1, x_2, \dots, x_n$$, we find a value that is a measure of centrality in a sense that the sum of all deviations (where each deviation is defined as $$u_i = x_i - \bar{x}$$) to the right of the mean value are equal to the sum of all the deviations to the left of that mean. There is no inherent reason why this measure is good, let alone the best way to describe the mean of a sample, but it is certainly intuitive and practical. The important point is, that by defining the arithmetic mean in this way, it necessarily follows that once we constructed the arithmetic mean, all deviations from that mean must sum to zero by definition!

In linear regression, this is no different. We fit the line such that the sum of all differences between our fitted values (which are on the regression line) and the actual values that are above the line is exactly equal to the sum of all differences between the regression line and all values below the line. Again, there is no inherent reason, why this is the best way to construct a fit, but it is straightforward and intuitively appealing. Just as with the arithmetic mean: by constructing our fitted values in this way, it necessarily follows, by construction, that all deviations from that line must sum to zero for otherwise this just wouldn't be an OLS regession.

• +1 for straightforward, simple and intuitive answer!
– user218970
Oct 7, 2018 at 9:40
• Great explanation, but I'm not sure, "Again, there is no inherent reason, why this is the best way to construct a fit, but it is straightforward and intuitively appealing." is accurate. It is well known by the Gauss-Markov Theorem that OLS estimators are BLUE: best (minimum-variance) linear unbiased estimates (assuming assumptions are met). Often, our intuitive "feelings" about what is appealing/reasonable is also backed up mathematically, as is the case here.
– Meg
Mar 22, 2020 at 15:11
• The explanation is geometrically appealing but fails to address the asked question about the necessity (or not) of an intercept term. May 16, 2022 at 19:31
• Perfect, thank you. And it makes sense. Oct 27, 2023 at 20:57

When an intercept is included in multiple linear regression, $$\hat{y}_i = \beta_0 + \beta_1x_{i,1} + \beta_2x_{i,2} +…+ \beta_px_{i,p}$$ In Least squares regression, the sum of the squares of the errors is minimized. $$SSE=\displaystyle\sum\limits_{i=1}^n \left(e_i \right)^2= \sum_{i=1}^n\left(y_i - \hat{y_i} \right)^2= \sum_{i=1}^n\left(y_i -\beta_0- \beta_1x_{i,1}-\beta_2x_{i,2}-…- \beta_px_{i,p} \right)^2$$ Take the partial derivative of SSE with respect to $$\beta_0$$ and setting it to zero. $$\frac{\partial{SSE}}{\partial{\beta_0}} = \sum_{i=1}^n 2\left(y_i -\beta_0- \beta_1x_{i,1}-\beta_2x_{i,2}-…- \beta_px_{i,p} \right)^1 (-1) =-2\displaystyle\sum\limits_{i=1}^ne_i=0$$ Hence, the residuals always sum to zero when an intercept is included in linear regression.

Write the linear model in matrical form as \begin{align} y = X\beta + \varepsilon, \end{align} where $$y \in \mathbb{R}^n$$ is the response vector, $$X \in \mathbb{R}^{n \times p}$$ is the design matrix, $$\varepsilon \in \mathbb{R}^n$$ is the error vector.

A key observation is that because the model has intercept, $$1$$, which is the first column of design matrix $$X$$, can be written as $$1 = Xe,$$ where $$e$$ is a column vector with all zeros but the first component one. Also note, in matrix notation, the sum of residuals is just $$1^T(y - \hat{y})$$, where $$\hat{y} = Hy$$ and $$H$$ is the projection matrix (i.e., "hat matrix") equals to $$X(X^TX)^{-1}X^T$$.

Therefore, \begin{align} & 1^T(y - \hat{y}) = 1^T(I - H)y \\ = & e^TX^T(I - X(X^TX)^{-1}X^T)y \\ = & e^T(X^T - X^TX(X^TX)^{-1}X^T)y \\ = & e^T(X^T - X^T)y \\ = & 0. \end{align}

A simple derivation using matrix algebra:

$\sum e$ can be written as $1^Te$

Then

$1^Te = 1^T(M_x y)$ where $M_x$ is the orthogonal matrix. Since $M_x$ is symmetric we can rearrange so that $(M_x1)^Ty$

which equals zero if $M_x$ and $1$ are orthogonal, which is the case if the matrix of the regressors $x$ contains the intercept (a vector of $1$, indeed).

1. $$e_i = y_i - [1, X] [a, b] = y_i - Xb - a = v_i - a$$
2. $$\frac{d}{da} \sum e_i^2 \propto \sum e_i\cdot 1 = \sum v_i - a = 0$$ so $$\hat{a} = \frac{1}{n}\sum v_i$$
3. $$\sum e_i = \sum_i v_i - a = \sum_i v_i - \frac{n}{n}\sum_i v_i = 0$$

..