Assumptions behind simple linear regression model [duplicate]

Let's say we have a simple linear regression model, that is, $y = X\beta + r$ where $y$ is a vector of size n x 1, $X$ a matrix of size n x p, $\beta$ the regression coefficient vector of size p x 1 and $r$ is a residual error of size n x 1.

Most of papers I read consider that $y$ follows a multivariate normal distribution. My question is whether $y$ can follow a distribution different from normal, for example elliptical, etc. Is it necessary that in linear regression, for $y$ to be assumed Gaussian?

I discussed with a researcher and he told me that linear regression models are only suitable (or efficient) for Gaussian assumption. Is this true?

• – Tim
Jun 30, 2015 at 13:53
• If your errors are iid normal, then yes, linear regression will give efficient estimates. But it's not necessary to have normality to use ordinary least squares linear regression. Jul 5, 2015 at 9:05
• Thank you so much for your answer, so OLS is efficient and robust? so OLS can be considered distribution free? Jul 5, 2015 at 9:36

1 Answer

When error-terms are non-normal, you still get many nice properties. The biggest of which is the Gauss-Markov theorem that tells you that the Ordinary Least Square (OLS) estimator, $\hat{\beta}_{OLS} = (X^TX)^{-1}X^TY$, is the best linear unbiased estimator (BLUE).

However, it is only due to the Normality of the error terms that $\hat{\beta}_{OLS}$ is also the Maximum Likelihood Estimator (MLE). That means, you cannot use the asymptotic properties of the MLE. In layman's terms, this means that you cannot use p-values and confidence intervals for $\hat{\beta}_{OLS}$ when the error terms are grossly non-Normal.

In summary, if the error terms are some non-Normal distribution you can use the point estimates for $\hat{\beta}_{OLS}$ but cannot interpret the p-values and confidence intervals that come from MLE theory.

• thank you for your answer. So using OLS estimator is robust under all assumptions? does OLS is only preferred in Gaussian hypothesis? Jun 30, 2015 at 22:53
• The OLS estimator is the best unbiased, linear, estimator. That is, you can interpret it's value but, depending on how non-Normal the error terms are, I would caution against interpeting any p-values or confidence intervals associated with it. Jul 1, 2015 at 0:26