As I understand, in simple linear regression, we have two options:
- OLS: OLS estimator is BLUE: Best (Lowest Variance) Linear Unbiased Estimator ... also normally distributed when sample size is large and consistent.
- MLE: MLE estimator has same properties: unbiased (but not for variance), also consistent (when likelihood is correctly chosen), also normally distributed when sample size is large, also lowest variance (cramer-rao)
But it seems that MLE is more popular in OLS in more advanced classes. In GLM, majority of estimation is done with MLE (I have never heard of OLS being used in GLM).
I read other posts on this (What are the properties of MLE that make it more desirable than OLS? Estimating linear regression with OLS vs. ML), but I can't find exact answer and reference. Both OLS and MLE look similar, but it seems MLE has more advantages.
I feel there is some tradeoff : MLE better (ex: stronger? lower bias?) when you have strong confidence in likelihood choice and is there for more sensitives , OLS is better when you have low confidence in likelihood choice and is therefore less sensitive.
Is there some exact way to understand comparative advantages of OLS vs MLE, when to use them and why MLE is more popular than OLS in higher classes? EX: is there some graph which shows strength of MLE vs strength of OLS for different sample sizes, wrong choice in likelihood, etc? Before doing analysis, is there some equation I can use for cost-benefit analysis that shows me the hypotethical gains from using MLE vs hypothethical loss from using MLE?