0
$\begingroup$

Thanks to lots of helpful answers in the community, I figured that Least Absolute Deviations regression can give better estimations when the normality of residuals is violated (e.g. residuals following Laplace distribution).

Meanwhile, a thought also came to my mind that in general with large sample size the parametric assumption of residuals following normal distribution need not have to be fulfilled, due to the central limit theorem. Then, considering that research nowadays often manage a great amount of data, is it valid to assume that there are few cases where researcher have to use LAD due to the issue of normality?

  • In a similar vein, if sample size provides a good buffer to the problem of normality of error, then it perhaps seems techniques such as log-transformation of skewed variables won't be needed much also.
$\endgroup$

1 Answer 1

0
$\begingroup$

If $Y|X$ is normally distributed, OLS is the maximum likelihood estimator (MLE). If $Y|X$ is Laplace distributed, LAD is the MLE. MLE has some desirable features (as you put it yourself, better estimations); see e.g. Chan "Lecture 8: Properties of Maximum Likelihood Estimation (MLE)" (2015) for details. Therefore, you may well prefer MLE to other alternatives. E.g. you would prefer OLS over LAD for normally distributed $Y|X$ and LAD over OLS for Laplace distributed $Y|X$.

All of the above holds regardless of the sample size. Now consider a sample from a given population or a data generating process. The distribution of $Y|X$ itself does not change with sample size. What may change is the sampling distribution of estimators. However, this is not what we care about in the discussion above. The superiority of OLS over LAD or vice versa depends on the distribution of $Y|X$, not the sampling distribution of the estimators.

(The answer could be reformulated in terms of model errors/residuals in place of $Y|X$.)

$\endgroup$
2
  • $\begingroup$ Thanks, great reply! One more follow-up question, - As you said, if I may reformulate the above in terms of 'residuals', then I guess the 'distribution of residuals' is given after the model selection between OLS & LAD. In other words, put it more easily, won't I have to decide between OLS & LAD and draw a regression line BEFORE I get my residuals realized? $\endgroup$
    – HYL
    Commented Feb 12, 2021 at 7:08
  • $\begingroup$ @Hwi-youngLee, realistically, you would not know the distribution of residuals until you have fit a model. So you could fit one with OLS and one with LAD and see what you get. If you get normal residuals from OLS, choose OLS. If you get Laplace residuals from LAD, choose LAD. Hopefully you do not get both. If you get neither, then neither OLS nor LAD would be MLEs. Then some more thought and experimentation with distributions and estimators is needed. $\endgroup$ Commented Feb 12, 2021 at 8:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.