MLE in context: why is maximum likelihood estimation a thing? I am trying to understand maximum likelihood estimation and the role it plays in the context of machine learning.
I think MLE basically says: Find such parameters so that the model best describes the data you have.
This sounds like a no-brainer to me. I can't think of any other parameters one could possibly be looking for. Clearly, I am missing something. Can MLE be put in contrast with any other method of estimating model parameters? Can one choose not to use MLE when doing machine learning?  
 A: Let's think about your characterization of what MLE is trying to do:
"Find such parameters so that the model best describes the data you have."
So, I feel that plugging in the best unbiased estimates of those parameters "best describes" the data. Could you blame me? But that's not MLE.
Using MLE for parameter estimation is not always the obvious thing to do. It may have poor finite sample size properties in your context, it may break down, or it may not even exist! And then there's the situation where you have prior information about your parameters.
Gauss in the 1800s used reasoning similar to maximum likelihood to derive the normal distribution (bottom of p. 103), a century before the concept was formalized by R.A. Fisher. It's likely that many other people used it some shape or form, and maybe you would have thought of it too, if you lived back then.
On the other hand, maximum likelihood kind of hacks the probability distribution in a way that might not have been obvious. The arguments of a probability distribution are values that the random variables can take, but maximum likelihood fixes those at sort of (but not completely) arbitrary values and then views that probability distribution as a function of totally different arguments, all to get an estimator. In all humility, I don't know that I would have thought of it.
