13
$\begingroup$

When is it preferable to use Maximum Likelihood Estimation instead of Ordinary Least Squares? What are the strengths and limitations of each? I am trying to gather practical knowledge on where to use each in common situations.

$\endgroup$
2
$\begingroup$

As explained here, OLS is just a particular instance of MLE. Here is closely related question, with a derivation of OLS in terms of MLE.

The conditional distribution corresponds to your noise model (for OLS: Gaussian and the same distribution for all inputs). There are other options (t-Student to deal with outliers, or allow the noise distribution to depend on the input)

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ (-1) It's not true that the student $t$ can be depended upon to deal with the outliers. See the example here or chapter 2 of this textbook for more explanations. $\endgroup$ – user603 Feb 16 '14 at 10:58
  • 8
    $\begingroup$ The OLS is a distance-minimizing approximation/estimation method, while ML is a "likelihood" maximization method. OLS needs no stochastic assumptions to provide his distance-minimizing solution, while ML starts by assuming a joint probability density/mass function. The fact that in some circumstances the two provide the same solution, in no way does it make the one a particular instance of the other. $\endgroup$ – Alecos Papadopoulos Feb 16 '14 at 14:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.