Consider the following simple linear regression model involving the $\epsilon_i$ error term,

$$y_i = \alpha + \beta x_i + \epsilon_i$$

such that,

$$\epsilon_i \sim \mathcal N(0,\sigma^2)$$

we know that if $Z_1, Z_2, ..., Z_n$ are independent, standard normal random variables,

$$Z_i \sim \mathcal N(0,1)$$

then the sum of their squares is distributed according to the chi-square distribution with $n$ degrees of freedom,

$$\sum _{i=1}^{n} Z_{i}^{2} \sim \chi^{2}_{n}$$

Now let's prove that $\frac{(n-2)s^2}{\sigma^2}\sim \chi^{2}_{n-2}$ such that,

$$s^2 = \frac{\sum _{i=1}^{n} \hat{\epsilon}_i^2}{n-2}$$



$$\epsilon_i \sim \mathcal N(0,\sigma^2)$$

$$\frac{\epsilon_i - \cancelto{0}{\overline{\epsilon}}}{\sigma} \sim \mathcal N(0, 1)$$

$$\sum _{i=1}^{n} \left(\frac{\epsilon_i}{\sigma}\right)^2 \sim \chi^{2}_{n}$$

$$\sum _{i=1}^{n} \frac{\epsilon_i^2}{\sigma^2} \sim \chi^{2}_{n}$$

How do I now move from this expression $\sum _{i=1}^{n} \frac{\epsilon_i^2}{\sigma^2}$ which uses the error term $\epsilon_i$ to this expression $\sum _{i=1}^{n} \frac{\hat{\epsilon}_i^2}{\sigma^2}$ which uses the residual term $\hat{\epsilon}_i$?


I came across this answer but I got lost in the second part of the demonstration and I wanted to know if there is a simpler method that would lead to the same answer.

  • 1
    $\begingroup$ stats.stackexchange.com/questions/20227/… $\endgroup$ Commented Nov 29, 2019 at 13:43
  • $\begingroup$ I think you need to assume the residuals are independent, though I have no idea how to go about showing they are. $\endgroup$
    – MSIS
    Commented Jan 31 at 0:22

1 Answer 1


Without using the orthogonal change of variables in the linked answer, you can work under the general matrix setup of multiple linear regression. Here we are concerned with ordinary least squares.

The key result to be used here is the Fisher-Cochran theorem on distribution of quadratic forms (e.g. see page 185-186, 2nd edition of Linear Statistical Inference and Its Applications by Rao).

Suppose the model is $$y_i=\beta_0+\beta_1x_i+\varepsilon_i$$

Define the vectors $y=(y_1,\ldots,y_n)$, $\beta=(\beta_0,\beta_1)$ and $\varepsilon=(\varepsilon_1,\ldots,\varepsilon_n)$.

So in matrix form you have $$y=X\beta+\varepsilon\,,$$ where $X_{n\times k}$ with rank $k$ (here $k=2$) is the matrix of covariates (fixed) and $\varepsilon\sim N(0,\sigma^2 I_n)$.

By standard algebra of least squares, $$\hat y=X\hat\beta=X(X'X)^{-1}X'y=Hy\,,$$ where $H=X(X'X)^{-1}X'$ is the hat matrix. Note that $H$ is symmetric and idempotent.

The residual vector is

\begin{align} e=y-\hat y&=(I_n-H)y \\&=(I_n-H)(X\beta+\varepsilon) \\&=(I_n-H)\varepsilon \end{align}

Now $I_n-H$ is symmetric and idempotent because $H$ is so.

So the residual sum of squares is

\begin{align} e'e&=y'(I_n-H)'(I_n-H)y \\&=y'(I_n-H)y \\&=\varepsilon'(I_n-H)\varepsilon \end{align}

Therefore $$\frac{(n-2)s^2}{\sigma^2}=\frac{e'e}{\sigma^2}=\left(\frac{\varepsilon}{\sigma}\right)'(I_n-H)\frac{\varepsilon}{\sigma}$$

By a corollary of the Fisher-Cochran theorem, the above has a chi-square distribution because $(I_n-H)$ is idempotent, the degrees of freedom being $\operatorname{rank}(I_n-H)$.

As rank of an idempotent matrix equals its trace, $\operatorname{rank}(I_n-H)=\operatorname{tr}(I_n-H)=n-\operatorname{tr}(H)$.

And by properties of trace, $\operatorname{tr}(H)=\operatorname{tr}(X(X'X)^{-1}X')=\operatorname{tr}(X'X(X'X)^{-1})=\operatorname{tr}(I_k)=k$

  • $\begingroup$ Hi @StubbornAtom, if you don't mind, in the residual vector step why is $(I_n-H)(X\beta+\varepsilon)=(I_n-H)\varepsilon$? How is $(X\beta+\varepsilon)=\varepsilon$? Thanks in adavnce. $\endgroup$ Commented Apr 29, 2020 at 17:43
  • 1
    $\begingroup$ Sure. $X\beta+\varepsilon=\varepsilon$ is obviously not true. You have $HX=X$ and \begin{align} (I_n-H)(X\beta+\varepsilon)&=(I_n-H)X\beta+(I_n-H)\varepsilon \\&=X\beta-HX\beta+(I_n-H)\varepsilon \end{align} $\endgroup$ Commented Apr 29, 2020 at 17:56
  • $\begingroup$ Thank you so much @StubbornAtom. $\endgroup$ Commented Apr 29, 2020 at 18:08
  • 1
    $\begingroup$ The theorem specifically used is a special case of the one here: stats.stackexchange.com/q/188626/119261. $\endgroup$ Commented Jul 24, 2020 at 17:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.