0
$\begingroup$

Let's say we have a simple linear regression model, that is, $y = X\beta + r$ where $y$ is a vector of size n x 1, $X$ a matrix of size n x p, $\beta$ the regression coefficient vector of size p x 1 and $r$ is a residual error of size n x 1.

Most of papers I read consider that $y$ follows a multivariate normal distribution. My question is whether $y$ can follow a distribution different from normal, for example elliptical, etc. Is it necessary that in linear regression, for $y$ to be assumed Gaussian?

I discussed with a researcher and he told me that linear regression models are only suitable (or efficient) for Gaussian assumption. Is this true?

Improve this question
$\endgroup$
3
  • 2
    $\begingroup$ Related: stats.stackexchange.com/questions/29731/… $\endgroup$
    – Tim
    Commented Jun 30, 2015 at 13:53
  • 1
    $\begingroup$ If your errors are iid normal, then yes, linear regression will give efficient estimates. But it's not necessary to have normality to use ordinary least squares linear regression. $\endgroup$
    – Glen_b
    Commented Jul 5, 2015 at 9:05
  • $\begingroup$ Thank you so much for your answer, so OLS is efficient and robust? so OLS can be considered distribution free? $\endgroup$
    – Christina
    Commented Jul 5, 2015 at 9:36

1 Answer 1

Reset to default
1
$\begingroup$

When error-terms are non-normal, you still get many nice properties. The biggest of which is the Gauss-Markov theorem that tells you that the Ordinary Least Square (OLS) estimator, $\hat{\beta}_{OLS} = (X^TX)^{-1}X^TY$, is the best linear unbiased estimator (BLUE).

However, it is only due to the Normality of the error terms that $\hat{\beta}_{OLS}$ is also the Maximum Likelihood Estimator (MLE). That means, you cannot use the asymptotic properties of the MLE. In layman's terms, this means that you cannot use p-values and confidence intervals for $\hat{\beta}_{OLS}$ when the error terms are grossly non-Normal.

In summary, if the error terms are some non-Normal distribution you can use the point estimates for $\hat{\beta}_{OLS}$ but cannot interpret the p-values and confidence intervals that come from MLE theory.

Improve this answer
$\endgroup$
2
  • $\begingroup$ thank you for your answer. So using OLS estimator is robust under all assumptions? does OLS is only preferred in Gaussian hypothesis? $\endgroup$
    – Christina
    Commented Jun 30, 2015 at 22:53
  • $\begingroup$ The OLS estimator is the best unbiased, linear, estimator. That is, you can interpret it's value but, depending on how non-Normal the error terms are, I would caution against interpeting any p-values or confidence intervals associated with it. $\endgroup$
    – TrynnaDoStat
    Commented Jul 1, 2015 at 0:26

Not the answer you're looking for? Browse other questions tagged or ask your own question.