I know that the residual variance can be defined like this: $$s_R^2=\frac{1}{n-2}\sum_{i=1}^ne_i^2$$ Apparently the denominator of the fraction is $n-2$ rather than $n$ because the $e_i$ have two linear restrictions: $$\overline{e}=\frac{1}{n}\sum_{i=1}^ne_i = 0 \quad , \quad \sum_{i=1}^n x_ie_i=0$$ While the first restriction seems natural to me, I don't understand where does the second restriction come from, nor why does that formula work. Could someone help me undestand this?

  • $\begingroup$ The second equality ensures that the $x_i$ and $e_i$ are uncorrelated (check out their sample covariance). The errors are considered independant and identically distributed, and the residuals have zero mean (as a result of OLS). $\endgroup$
    – chl
    Oct 14 '20 at 11:10

Both restrictions are properties of regression hyperplane. They are easily proven after you minimise the square sum of residuals:

$\mathbf{X}'\mathbf{y} = \mathbf{X}'\mathbf{Xb}$ can be solved to obtain regression coefficient $\mathbf{b}=(\mathbf{X}'\mathbf{X})^{-1}\mathbf{X}'\mathbf{y}$, or to prove second (in yours description) property of regression hyperplane:

$$\mathbf{X}'\mathbf{y} = \mathbf{X}'\mathbf{Xb}$$ $$\mathbf{X}'\mathbf{y} - \mathbf{X}'\mathbf{Xb} = \mathbf{0}$$ $$\mathbf{X}'(\mathbf{y} - \mathbf{Xb}) = \mathbf{0}$$ $$\mathbf{X}'\mathbf{e} = \mathbf{0}$$

There is whole matrix $\mathbf{X}$ in this equation, and therefore this is a set of equations.

However in a model with a constant, the first column of matrix $\mathbf{X}$ is equal to vector of ones. Therefore if we pick only first equation of the whole set:

$$[1,...,1]\cdot[e_1,...,e_n]' = 0 $$ $$\sum_{i=1}^{n}e_i = 0$$

Again, as they are properties, they are always true if you use OLS (and a model with constant for the first described by you property).

The character $'$ stands for Transmutation; bold stands for matrix and vector notation.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.