0
$\begingroup$

The Wikipedia entry for Degrees of freedom (statistics) has a section, "Of residuals," that discusses the two equations that constrain the residuals:

$\hat{e_1} + ... + \hat{e_n} = 0$ and

$x_1\hat{e_1} + ... + x_n\hat{e_n} = 0$

The first equation is obvious. I am struggling to understand why the second equation must be true.

Improve this question
$\endgroup$
2
  • $\begingroup$ Please see stats.stackexchange.com/…. $\endgroup$
    – whuber ♦
    Jun 5, 2019 at 1:06
  • $\begingroup$ Thanks. Should I delete the question since it's a duplicate? $\endgroup$
    – Jon
    Jun 5, 2019 at 1:15

1 Answer 1

Reset to default
0
$\begingroup$

I'm able to show that the second equation is true using the alternative formulas for $Sxx$ and $Sxy$ (see below), but still hoping for a more intuitive explanation for why the sum of those products must be zero.

Given:

$Sxy = \Sigma x_iy_i - (\Sigma x_i)(\Sigma y_i)/n$

$Sxx = \Sigma x_i^2 -(\Sigma x_i)^2/n$

Least squares estimate of the slope: $b_1 = Sxy / Sxx$

Least squares estimate of the intercept: $b_0 = \overline{y} - b_1 \overline{x} = \Sigma{y_i}/n - b_1 \Sigma{x_i}/n$

Now,

$x_1 \hat{e_1} + ... + x_n \hat{e_n} = x_1(y_1 - (b_0 + b_1x_1)) + ... + x_n(y_n - (b_0 + b_1x_n)) = $

$x_1y_1 + ... + x_ny_n - b_0(x_1+...+x_n) - b_1(x_1^2 + ... + x_n^2) = $

$\Sigma{x_i y_i} - b_0 \Sigma{x_i} - b_1 \Sigma{x_i^2} = $

$\Sigma{x_i y_i} - (\Sigma{y_i}/n - b_1 \Sigma{x_i}/n)\Sigma{x_i} -b_1\Sigma{x_1^2} = $

$\Sigma{x_i y_i} - \Sigma{x_i}\Sigma{y_i}/n + b_1((\Sigma{x_i})^2/n - \Sigma{x_i^2}) =$

$\Sigma{x_i y_i} - \Sigma{x_i}\Sigma{y_i}/n + \frac{\Sigma{x_i y_i} - \Sigma{x_i}\Sigma{y_i}/n}{\Sigma{x_i^2} - (\Sigma{x_i})^2/n}[(\Sigma{x_i})^2/n - \Sigma{x_i^2}] = $

$\Sigma{x_i y_i} - \Sigma{x_i}\Sigma{y_i}/n - (\Sigma{x_i y_i} - \Sigma{x_i}\Sigma{y_i}/n ) = 0$

Improve this answer
$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.