Deriving SSE of Simple Linear Regression is $\chi^{2}$ As per my notes, the key step in the proof that the sum of squares of residuals in regression is $\chi^{2}_{n-2}$ is the fact that $e_{i} = y_{i} - \hat{y}_{i}$ has a mean 0 and variance $\sigma^{2}$. This is the reason why $\dfrac{e_{i}}{\sigma}$ is standard normal and the sum of squares of the residuals turns out to be $\chi^{2}_{n-2}$.
The problem I have with this is that I derived the variance of a residual as $\sigma^{2}[1 - \frac{1}{n} - \frac{(x_{i} - \overline{x})^{2}}{S_{xx}}]$ when I was doing the predicted value of a new observation using the model.
Where is the logical flaw in the proof that the sum of squares of residuals is $\chi^{2}_{n-2}$ and how do we derive the fact that $\sum_{i=1}^{n}(\frac{e_i^{2}}{\sigma^2})$ is $\chi^{2}_{n-2}$?
 A: Okay, so it turns out that the answer is not at all obvious as it was made out to be in the lecture videos I was watching (nice set of videos posted here ). We can solve it for the general case of multiple linear regression like so:
The model is $ \mathbf{Y} = \mathbf{X}\mathbf{\beta} + \mathbf{\epsilon}$. The fitted values are $ \hat{\mathbf{Y}} = \mathbf{X}\hat{\beta} = \mathbf{HY}$, where $\mathbf{H}$ is the hat matrix. From the above hat matrix notation, $\mathbf{e = Y - \hat{Y} = (I - H)Y = (I - H)(X\beta+\epsilon) = (I - H)\epsilon}$
The sum of the residual squares,
$ SSE = \sum_{i=1}^{n} e_{i}^{2} = \mathbf{e^{'} e = \epsilon^{'}(I - H)\epsilon}$. This follows from the symmetric nature and the idempotence of $\mathbf{(I - H)}$.
So, $\dfrac{SSE}{\sigma^{2}} = \dfrac{\mathbf{\epsilon^{'}(I - H)\epsilon}}{\sigma^{2}} = \mathbf{\dfrac{\epsilon^{'}}{\sigma} (I - H) \dfrac{\epsilon}{\sigma}}$.
This final formula is now in the quadratic form $\mathbf{X = \dfrac{Y^{'}MY}{\sigma^{2}}}$, where $\mathbf{Y \sim N(0, \sigma^{2}I})$ and $\mathbf{M}$ is symmetric and idempotent. Using the properties of orthogonal factorization of M, we can show that the distribution of $\mathbf{X}$ is $\chi^{2}_{tr(M)}$, which completes this rather neat proof.
Much of the needed algebra was inspired by the answers to this question - In simple linear regression, where does the formula for the variance of the residuals come from?
