1
$\begingroup$

In my notes from university I have written down that the residual standard error (from normal linear regression) has the following distribution

$\frac{\hat{\sigma}^2}{\sigma^2}\sim \frac{\chi^{2}_{n-p}}{n-p}$

I do not understand how this derived though.

We know
$\hat{\sigma}^2 = \frac{1}{n-p} \sum_{i=1}^n{e_i^2}$

e being the residuals of the model and
$e \sim N(0,\sigma^2M)$
$ M = I_n - X(X^tX)^{-1}X^t$

also

$\chi^2_n = \sum_{i=1}^n{N(0,1)^2}$

So I could understand the result if $e \sim N(0,\sigma^2)$ and the degrees of freedom are n. However the formula does not appear to standardise the M part of the variance.

I believe it has something to do with M being a idempotent matrix of rank n-p but I don't understand how that can be used do derive the above mentioned result.

any help would be greatly appreciated.

$\endgroup$
  • 1
    $\begingroup$ Hint: the trace of any idempotent matrix equals its rank. Now compare the role of "$n-p$" in its first appearance in this post to its role in its last appearance. $\endgroup$ – whuber Feb 14 '16 at 12:34
0
$\begingroup$

After playing around with this a bit more I've got the following, though I'm not sure the matrix algebra is entirely accurate but it seems to make sense.

$e \sim N_n(0 , \sigma^2M) $

thus

$\sigma^{-1} M^{\frac{-1}{2}}e \sim N_n(0,I_n)$ and
$\sigma^{-1} e^tM^{\frac{-1}{2}} \sim N_n(0,I_n)$

as $M^t = M$ and $\sigma$ is a constant

We have in our notes (without proof) that

$x^t K x \sim \chi^2_r$

Where K is an idempotent matrix of rank r , $x \sim N_n(\mu,\sigma^2 I_n)$ and $K\mu = 0$

Therefore

$\sum{e_i^2\sigma^{-2}} = $
$\sigma^{-2}e^te = $
$\sigma^{-2}e^tM^{-\frac{1}{2}}MM^{-\frac{1}{2}}e $

$ (e^t\sigma^{-1} M^{\frac{-1}{2}}) * M * (\sigma^{-1} M^{\frac{-1}{2}}e) \sim \chi^2_{n-p}$

as M is rank n-p

thus $\frac{\hat{\sigma}^2}{\sigma^2} \sim \frac{\chi^2_{n-p}}{n-p} $

$\endgroup$
  • $\begingroup$ Is anyone able to help? If I'm not getting replies due to the way I've structured the question / answer please let me know so I can rework it $\endgroup$ – gowerc Feb 19 '16 at 9:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.