Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

When is it preferable to use Maximum Likelihood Estimation instead of Ordinary Least Squares? What are the strengths and limitations of each? I am trying to gather practical knowledge on where to use each in common situations.

share|improve this question
add comment

1 Answer 1

As explained here, OLS is just a particular instance of MLE. Here is closely related question, with a derivation of OLS in terms of MLE.

The conditional distribution corresponds to your noise model (for OLS: Gaussian and the same distribution for all inputs). There are other options (t-Student to deal with outliers, or allow the noise distribution to depend on the input)

share|improve this answer
    
(-1) It's not true that the student $t$ can be depended upon to deal with the outliers. See the example here or chapter 2 of this textbook for more explanations. –  user603 Feb 16 at 10:58
    
The OLS is a distance-minimizing approximation/estimation method, while ML is a "likelihood" maximization method. OLS needs no stochastic assumptions to provide his distance-minimizing solution, while ML starts by assuming a joint probability density/mass function. The fact that in some circumstances the two provide the same solution, in no way does it make the one a particular instance of the other. –  Alecos Papadopoulos Feb 16 at 14:35
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.