0
$\begingroup$

As I understand, in simple linear regression, we have two options:

  • OLS: OLS estimator is BLUE: Best (Lowest Variance) Linear Unbiased Estimator ... also normally distributed when sample size is large and consistent.
  • MLE: MLE estimator has same properties: unbiased (but not for variance), also consistent (when likelihood is correctly chosen), also normally distributed when sample size is large, also lowest variance (cramer-rao)

But it seems that MLE is more popular in OLS in more advanced classes. In GLM, majority of estimation is done with MLE (I have never heard of OLS being used in GLM).

I read other posts on this (What are the properties of MLE that make it more desirable than OLS? Estimating linear regression with OLS vs. ML), but I can't find exact answer and reference. Both OLS and MLE look similar, but it seems MLE has more advantages.

I feel there is some tradeoff : MLE better (ex: stronger? lower bias?) when you have strong confidence in likelihood choice and is there for more sensitives , OLS is better when you have low confidence in likelihood choice and is therefore less sensitive.

Is there some exact way to understand comparative advantages of OLS vs MLE, when to use them and why MLE is more popular than OLS in higher classes? EX: is there some graph which shows strength of MLE vs strength of OLS for different sample sizes, wrong choice in likelihood, etc? Before doing analysis, is there some equation I can use for cost-benefit analysis that shows me the hypotethical gains from using MLE vs hypothethical loss from using MLE?

Improve this question
$\endgroup$
6
  • 2
    $\begingroup$ OLS coincides with MLE for independent and equal-variance conditional Gaussian distributions. $\endgroup$
    – Dave
    Commented Oct 31, 2023 at 2:53
  • $\begingroup$ I wish there was some equation to understand the advantages of mle vs ols: e.x. "optimality of mle dvivided by optimality of ols" increases when these conditions are present .... and decreases when these conditions are present? $\endgroup$
    – stats_noob
    Commented Oct 31, 2023 at 2:55
  • $\begingroup$ This is a kind of either/or fallacy. In terms of classroom exposure, MLE is typically introduced in the probability and statistics sequence whereas there are separate courses that focus on regression with prob stat as a preliminary. $\endgroup$
    – AdamO
    Commented Oct 31, 2023 at 4:39
  • $\begingroup$ Are you sure MLE is unbiased in linear regression? In general, MLE is not unbiased, but I have forgotten if linear regression is an exception to that. $\endgroup$
    – Richard Hardy
    Commented Nov 4, 2023 at 6:41
  • $\begingroup$ @RichardHardy Gauss-Markov theorem, at least for a Gaussian likelihood $\endgroup$
    – Dave
    Commented Nov 4, 2023 at 13:44

1 Answer 1

Reset to default
5
$\begingroup$

OLS works only for one specific configuration of GLM. For other configurations MLE can be applied to estimate parameters. When you write estimation software you could switch to OLS for that one case, but there's no benefit for doing so. It's not like people care that much for assumptions when going after coefficients. Plus, you can relatively easily switch the distributions with MLE

Improve this answer
$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.