Given the Linear Regression model $y=X\beta+\epsilon$, where $\epsilon \sim D(0_n,\sigma^2 I_n)$ and $D$ is some distribution. How to find the OLS estimator of $\sigma^2$.

I know that the sum of least-squares residuals has a distribution equal to $\sigma^2$ times $\chi^2_{n-1}$. So I can therefore determine the unbiased estimator of $\sigma^2$. But how to prove that sum of least-squares residuals follows such distribution. Also anyone with new way to find it are welcome.

  • $\begingroup$ In $D(0_n,\sigma^2 I_n)$, if $0_n$ is the mean and $\sigma^2$ = variance, which kind of distributions meet this condition, apart from normal? $\endgroup$ – user158565 Oct 9 '18 at 0:22

The OLS (Ordinary Least Squares) estimate does not depend on the distribution D, so for any distribution you can use the exact same tools as the for the normal distribution.

This just gives the OLS estimates of the parameters, it does not justify any tests or other inference that could depend on your distribution D (though the Central Limit Theorem holds for regression and for large enough sample sizes (how big depends on how non-normal D is) the normal based tests and inference will still be approximately correct.

If you want Maximum Likelihood estimation instead of OLS, then this will depend on D). The normal distribution has the advantage that OLS gives the Maximum Likelihood answer as well.

  • $\begingroup$ Yeah so what i have written is correct? i.e., RSS $\sim \sigma^2*\chi^2_{n-1}$. So, E(RSS)=$\sigma^2*(n-1)$ and thus we can derive the estimator for $\sigma^2$ right? I'm not sure of the $n-1$ part. $\beta$ isn't a scalar, so should it be $n-k$ where $\beta$ is $k\times 1$ matrix? $\endgroup$ – Aatsrh Oct 9 '18 at 2:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.