LSE vs MLE - why use LSE when we have MLE?

Question

I know that with linear regressions Least Squares Estimate (LSE) and Maximum Likelihood Estimate (MLE) have differences, in that (1) MLE does not use unbiased standard deviation(the denominator is n) (2) While LSE requires homogenous E(residuals|X) & V(residuals|X), MLE does not, (3) LSE and MLE get identical when the residuals of LSE follow N(0, sigma^2).

Considering that LSE requires further assumptions[(3)] to become MLE, it seems that MLE has more beneficial characteristics of being a statistical estimator. My question is, then, Why do we have to use LSE when we have MLE, since the computation on both coefficients doesn't seem to differ in their difficulty? Some colleagues of mine told me that deciding which to use differs by field and interests, but it seems to me that MLE serves better in nearly every aspect and at the same time computing MLE doesn't seem to be much more difficult. So, Why not MLE for every case rather than LSE?

Answer 1

As an overarching framework, indeed MLE is much better. It shows when you really want to use LSE, and when you want to use other estimators, such as LAE, Poisson regression, logistic regression, WLS, GLS, etc. It also leads naturally and seamlessly to Bayesian methods. All quantitative disciplines would be much better off to replace LSE with MLE.

These days, there is not much to motivate LSE other than "we've always done it this way." Back in the 19th century, computational considerations were important and justified LSE, but certainly not any more. Some like to appeal to the Gauss-Markov Theorem to justify LSE, but that theorem is nearly vacuous. It states that LSE is best among estimators that are unbiased and linear functions of the data, but since the class of linear, unbiased estimators is so limited, that theorem is akin to stating that LSE is the biggest fish in a tiny puddle.