This question could seem pretty common but I want to give it a different take.

As an undergraduate student, for estimation of parameters I was taught about the glorified Maximum Likelihood and Least Squares methods, that are very useful for many properties they yield. For instance, a strong factor for which people choose MLE is that it gives asymptotically efficient estimations.

I know there are historically a variety of alternative method, but what I am looking for are ones that currently find a niche of usage as better choices, not just for textbook-only never-used examples.

Also I wouldn't mind a not too in-depth answer, if the field of usage is clear for someone to further research for other informations and the advantages over the above mentioned methods are clearly described. It is also totally fine for different answers to add just one or two methods.

EDIT: I am mainly interested in non-Bayesian statistics that arise from physical experiments (projects on large scale such as CERN projects) or economic studies.


closed as too broad by whuber Jan 16 '18 at 23:43

Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. Avoid asking multiple distinct questions at once. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.

  • $\begingroup$ method of moments comes up a fair bit still. We have quite a few questions on site that discuss it. Other approaches that are used in practice include optimizing some goodness of fit criterion (such as Kplmogorov-Smirnov distance or minimum chi-square on discrete distributions) or method of quantiles (often used for robustness or because only some quantiles are available). Numerous others in different circumstances. You could fill pages with methods of estimation which have important niche uses. $\endgroup$ – Glen_b Jan 16 '18 at 23:41
  • 2
    $\begingroup$ This is awfully broad: you are asking for an account of the theory of estimation. It includes, but isn't limited to, MLE, linear unbiased estimators, minimax estimators, Bayes estimators (for any loss); direct comparison of loss functions; concepts of risk and Bayes risk; and many more. I could see how a question asking for principles of selecting statistical estimators might (sort of) fit our format, but this one demands a long list of different answers. That doesn't really work here. (cc @glen_b) $\endgroup$ – whuber Jan 16 '18 at 23:42
  • $\begingroup$ I agree, as it stands it's too broad. $\endgroup$ – Glen_b Jan 16 '18 at 23:46
  • 1
    $\begingroup$ Fair enough, I am mainly interested in non-Bayesian statistics that arise from physical experiments or economic studies.; I am sorry for not fitting your format, I'll edit it as soon as possible. Also what @whuber said is the follow-up thing I am mostly interested: principles for selecting statistical estimator over other ones. $\endgroup$ – John Jan 16 '18 at 23:51

Generalized Method of Moments (GMM) is used by hundreds of econometricians every day. In the limit, GMM may lose to MLE. However, in finite samples GMM oftentimes has better numerical properties.

In the modeling setting, regularization can be applied if the observation-to-parameter ratio is low. Notable examples are lasso and ridge regression. Regularization helps in reducing the variance term in the bias-variance tradeoff. This is done at the expense of introducing a (hopefully small) bias.


Not the answer you're looking for? Browse other questions tagged or ask your own question.