I know that the residual variance can be defined like this: $$s_R^2=\frac{1}{n-2}\sum_{i=1}^ne_i^2$$ Apparently the denominator of the fraction is $n-2$ rather than $n$ because the $e_i$ have two linear restrictions: $$\overline{e}=\frac{1}{n}\sum_{i=1}^ne_i = 0 \quad , \quad \sum_{i=1}^n x_ie_i=0$$ While the first restriction seems natural to me, I don't understand where does the second restriction come from, nor why does that formula work. Could someone help me undestand this?
1
-
$\begingroup$ The second equality ensures that the $x_i$ and $e_i$ are uncorrelated (check out their sample covariance). The errors are considered independant and identically distributed, and the residuals have zero mean (as a result of OLS). $\endgroup$– chlOct 14, 2020 at 11:10
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
Both restrictions are properties of regression hyperplane. They are easily proven after you minimise the square sum of residuals:
$\mathbf{X}'\mathbf{y} = \mathbf{X}'\mathbf{Xb}$ can be solved to obtain regression coefficient $\mathbf{b}=(\mathbf{X}'\mathbf{X})^{-1}\mathbf{X}'\mathbf{y}$, or to prove second (in yours description) property of regression hyperplane:
$$\mathbf{X}'\mathbf{y} = \mathbf{X}'\mathbf{Xb}$$ $$\mathbf{X}'\mathbf{y} - \mathbf{X}'\mathbf{Xb} = \mathbf{0}$$ $$\mathbf{X}'(\mathbf{y} - \mathbf{Xb}) = \mathbf{0}$$ $$\mathbf{X}'\mathbf{e} = \mathbf{0}$$
There is whole matrix $\mathbf{X}$ in this equation, and therefore this is a set of equations.
However in a model with a constant, the first column of matrix $\mathbf{X}$ is equal to vector of ones. Therefore if we pick only first equation of the whole set:
$$[1,...,1]\cdot[e_1,...,e_n]' = 0 $$ $$\sum_{i=1}^{n}e_i = 0$$
Again, as they are properties, they are always true if you use OLS (and a model with constant for the first described by you property).
The character $'$ stands for Transmutation; bold stands for matrix and vector notation.
