# Why is the dot product of x and e equal to zero with least squares regression? [duplicate]

The Wikipedia entry for Degrees of freedom (statistics) has a section, "Of residuals," that discusses the two equations that constrain the residuals:

$$\hat{e_1} + ... + \hat{e_n} = 0$$ and

$$x_1\hat{e_1} + ... + x_n\hat{e_n} = 0$$

The first equation is obvious. I am struggling to understand why the second equation must be true.

– whuber
Jun 5, 2019 at 1:06
• Thanks. Should I delete the question since it's a duplicate?
– Jon
Jun 5, 2019 at 1:15

I'm able to show that the second equation is true using the alternative formulas for $$Sxx$$ and $$Sxy$$ (see below), but still hoping for a more intuitive explanation for why the sum of those products must be zero.

Given:

$$Sxy = \Sigma x_iy_i - (\Sigma x_i)(\Sigma y_i)/n$$

$$Sxx = \Sigma x_i^2 -(\Sigma x_i)^2/n$$

Least squares estimate of the slope: $$b_1 = Sxy / Sxx$$

Least squares estimate of the intercept: $$b_0 = \overline{y} - b_1 \overline{x} = \Sigma{y_i}/n - b_1 \Sigma{x_i}/n$$

Now,

$$x_1 \hat{e_1} + ... + x_n \hat{e_n} = x_1(y_1 - (b_0 + b_1x_1)) + ... + x_n(y_n - (b_0 + b_1x_n)) =$$

$$x_1y_1 + ... + x_ny_n - b_0(x_1+...+x_n) - b_1(x_1^2 + ... + x_n^2) =$$

$$\Sigma{x_i y_i} - b_0 \Sigma{x_i} - b_1 \Sigma{x_i^2} =$$

$$\Sigma{x_i y_i} - (\Sigma{y_i}/n - b_1 \Sigma{x_i}/n)\Sigma{x_i} -b_1\Sigma{x_1^2} =$$

$$\Sigma{x_i y_i} - \Sigma{x_i}\Sigma{y_i}/n + b_1((\Sigma{x_i})^2/n - \Sigma{x_i^2}) =$$

$$\Sigma{x_i y_i} - \Sigma{x_i}\Sigma{y_i}/n + \frac{\Sigma{x_i y_i} - \Sigma{x_i}\Sigma{y_i}/n}{\Sigma{x_i^2} - (\Sigma{x_i})^2/n}[(\Sigma{x_i})^2/n - \Sigma{x_i^2}] =$$

$$\Sigma{x_i y_i} - \Sigma{x_i}\Sigma{y_i}/n - (\Sigma{x_i y_i} - \Sigma{x_i}\Sigma{y_i}/n ) = 0$$