1
$\begingroup$

In my notes from university I have written down that the residual standard error (from normal linear regression) has the following distribution

$\frac{\hat{\sigma}^2}{\sigma^2}\sim \frac{\chi^{2}_{n-p}}{n-p}$

I do not understand how this derived though.

We know
$\hat{\sigma}^2 = \frac{1}{n-p} \sum_{i=1}^n{e_i^2}$

e being the residuals of the model and
$e \sim N(0,\sigma^2M)$
$ M = I_n - X(X^tX)^{-1}X^t$

also

$\chi^2_n = \sum_{i=1}^n{N(0,1)^2}$

So I could understand the result if $e \sim N(0,\sigma^2)$ and the degrees of freedom are n. However the formula does not appear to standardise the M part of the variance.

I believe it has something to do with M being a idempotent matrix of rank n-p but I don't understand how that can be used do derive the above mentioned result.

any help would be greatly appreciated.

$\endgroup$
  • 1
    $\begingroup$ Hint: the trace of any idempotent matrix equals its rank. Now compare the role of "$n-p$" in its first appearance in this post to its role in its last appearance. $\endgroup$ – whuber Feb 14 '16 at 12:34
0
$\begingroup$

After playing around with this a bit more I've got the following, though I'm not sure the matrix algebra is entirely accurate but it seems to make sense.

$e \sim N_n(0 , \sigma^2M) $

thus

$\sigma^{-1} M^{\frac{-1}{2}}e \sim N_n(0,I_n)$ and
$\sigma^{-1} e^tM^{\frac{-1}{2}} \sim N_n(0,I_n)$

as $M^t = M$ and $\sigma$ is a constant

We have in our notes (without proof) that

$x^t K x \sim \chi^2_r$

Where K is an idempotent matrix of rank r , $x \sim N_n(\mu,\sigma^2 I_n)$ and $K\mu = 0$

Therefore

$\sum{e_i^2\sigma^{-2}} = $
$\sigma^{-2}e^te = $
$\sigma^{-2}e^tM^{-\frac{1}{2}}MM^{-\frac{1}{2}}e $

$ (e^t\sigma^{-1} M^{\frac{-1}{2}}) * M * (\sigma^{-1} M^{\frac{-1}{2}}e) \sim \chi^2_{n-p}$

as M is rank n-p

thus $\frac{\hat{\sigma}^2}{\sigma^2} \sim \frac{\chi^2_{n-p}}{n-p} $

$\endgroup$
  • $\begingroup$ Is anyone able to help? If I'm not getting replies due to the way I've structured the question / answer please let me know so I can rework it $\endgroup$ – gowerc Feb 19 '16 at 9:54

Not the answer you're looking for? Browse other questions tagged or ask your own question.