Maximum Likelihood Estimation -- why it is used despite being biased in many cases Maximum likelihood estimation often results into biased estimators (e.g., its estimate for the sample variance is biased for the Gaussian distribution).
What then makes it so popular? Why exactly is it used so much? Also, what in particular makes it better than the alternative approach -- method of moments?
Also, I noticed that for the Gaussian, a simple scaling of the MLE estimator makes it unbiased. Why is this scaling not a standard procedure? I mean -- Why is it that after MLE computation, it is not routine to find the necessary scaling to make the estimator unbiased? The standard practice seems to be the plain computation of the MLE estimates, except of course for the well known Gaussian case where the scaling factor is well known.
 A: I'd add that sometimes (often) we use an MLE estimator because that's what we got, even if in an ideal world it wouldn't be what we want. (I often think of statistics as being like engineering, where we use what we got, not what we want.) In many cases it's easy to define and solve for the MLE, and then get a value using an iterative approach. Whereas for a given parameter in a given situation there may be a better estimator (for some value of "better"), but finding it may require being very clever; and when you're done being clever, you still only have the better estimator for that one particular problem.
A: Unbiasedness isn't necessarily especially important on its own.
Aside a very limited set of circumstances, most useful estimators are biased, however they're obtained.
If two estimators have the same variance, one can readily mount an argument for preferring an unbiased one to a biased one, but that's an unusual situation to be in (that is, you may reasonably prefer unbiasedness, ceteris paribus -- but those pesky ceteris are almost never paribus).
More typically, if you want unbiasedness you'll be adding some variance to get it, and then the question would be why would you do that?
Bias is how far the expected value of my estimator will be too high on average (with negative bias indicating too low). 
When I'm considering a small sample estimator, I don't really care about that. I'm usually more interested in how far wrong my estimator will be in this instance - my typical distance from right... something like a root-mean-square error or a mean absolute error would make more sense. 
So if you like low variance and low bias, asking for say a minimum mean square error estimator would make sense; these are very rarely unbiased.
Bias and unbiasedness is a useful notion to be aware of, but it's not an especially useful property to seek unless you're only comparing estimators with the same variance.
ML estimators tend to be low-variance; they're usually not minimum MSE, but they often have lower MSE than than modifying them to be unbiased (when you can do it at all) would give you.
As an example, consider estimating variance when sampling from a normal distribution $\hat{\sigma}^2_\text{MMSE} = \frac{S^2}{n+1}, \hat{\sigma}^2_\text{MLE} = \frac{S^2}{n}, \hat{\sigma}^2_\text{Unb} = \frac{S^2}{n-1}$ (indeed the MMSE for the variance always has a larger denominator than $n-1$).
A: Maximum likelihood estimation (MLE) yields the most likely value of the model parameters, given the model and the data at hand -- which is a pretty attractive concept. Why would you choose parameter values that make the data observed less probable when you can choose the values that make the data observed the most probable across any set of values? Would you wish to sacrifice this feature for unbiasedness? I do not say the answer is always clear, but the motivation for MLE is pretty strong and intuitive.
Also, MLE may be more widely applicable than method of moments, as far as I know. MLE seems more natural in cases of latent variables; for example, a moving average (MA) model or a generalized autoregressive conditional heteroskedasticity (GARCH) model can be directly estimated by MLE (by directly I mean it is enough to specify a likelihood function and submit it to an optimization routine) -- but not by method of moments (although indirect solutions utilizing the method of moments may exist).
A: Actually, the scaling of the maximum likelihood estimates in order to obtain unbiased estimates is a standard procedure in many estimation problems. The reason for that is that the mle is a function of the sufficient statistics and so by the Rao-Blackwell theorem if you can find an unbiased estimator based on sufficient statistics, then you have a Minimum Variance Unbiased Estimator.
I know that your question is more general than that but what I mean to emphasize is that key concepts are intimately related to the likelihood and estimates based on it. These estimates might not be unbiased in finite samples but they are asymptotically so and moreover they are asymptotically efficient, i.e. they attain the Cramer-Rao bound of variance for unbiased estimators, which might not always be the case for the MOM estimators.
A: To answer your question of why the MLE is so popular, consider that although it can be biased, it is consistent under standard conditions. In addition, it is asymptotically efficient, so at least for large samples, the MLE is likely to do as well or better as any other estimator you may cook up. Finally, the MLE is found by a simple recipe; take the likelihood function and maximize it. In some cases, that recipe may be hard to follow, but for most problems, it is not. Plus, once you have this estimate, we can derive the asymptotic standard errors right away using Fisher's information. Without using the Fisher's information, it is often really hard to derive the error bounds.
This is why MLE estimation is very often the go to estimator (unless you're a Bayesian); it's simple to implement and likely to be just as good if not better than anything else you need to do more work to cook up. 
