# Deriving SSE of Simple Linear Regression is $\chi^{2}$

As per my notes, the key step in the proof that the sum of squares of residuals in regression is $\chi^{2}_{n-2}$ is the fact that $e_{i} = y_{i} - \hat{y}_{i}$ has a mean 0 and variance $\sigma^{2}$. This is the reason why $\dfrac{e_{i}}{\sigma}$ is standard normal and the sum of squares of the residuals turns out to be $\chi^{2}_{n-2}$.

The problem I have with this is that I derived the variance of a residual as $\sigma^{2}[1 - \frac{1}{n} - \frac{(x_{i} - \overline{x})^{2}}{S_{xx}}]$ when I was doing the predicted value of a new observation using the model.

Where is the logical flaw in the proof that the sum of squares of residuals is $\chi^{2}_{n-2}$ and how do we derive the fact that $\sum_{i=1}^{n}(\frac{e_i^{2}}{\sigma^2})$ is $\chi^{2}_{n-2}$?

• Please add the [self-study] tag & read its wiki. Jul 16, 2016 at 1:53
• Try to find the distribution of $e_i$.
– Stat
Jul 16, 2016 at 15:45
• stats.stackexchange.com/questions/20227/… Jan 18, 2020 at 19:09

The model is $\mathbf{Y} = \mathbf{X}\mathbf{\beta} + \mathbf{\epsilon}$. The fitted values are $\hat{\mathbf{Y}} = \mathbf{X}\hat{\beta} = \mathbf{HY}$, where $\mathbf{H}$ is the hat matrix. From the above hat matrix notation, $\mathbf{e = Y - \hat{Y} = (I - H)Y = (I - H)(X\beta+\epsilon) = (I - H)\epsilon}$
The sum of the residual squares, $SSE = \sum_{i=1}^{n} e_{i}^{2} = \mathbf{e^{'} e = \epsilon^{'}(I - H)\epsilon}$. This follows from the symmetric nature and the idempotence of $\mathbf{(I - H)}$.
So, $\dfrac{SSE}{\sigma^{2}} = \dfrac{\mathbf{\epsilon^{'}(I - H)\epsilon}}{\sigma^{2}} = \mathbf{\dfrac{\epsilon^{'}}{\sigma} (I - H) \dfrac{\epsilon}{\sigma}}$.
This final formula is now in the quadratic form $\mathbf{X = \dfrac{Y^{'}MY}{\sigma^{2}}}$, where $\mathbf{Y \sim N(0, \sigma^{2}I})$ and $\mathbf{M}$ is symmetric and idempotent. Using the properties of orthogonal factorization of M, we can show that the distribution of $\mathbf{X}$ is $\chi^{2}_{tr(M)}$, which completes this rather neat proof.