When trying to estimate the parameters of a known distribution, it might occur that the maximum likelihood estimator and the method of moment estimator don't work well when there's only one observation.

What are some good non-Bayesian alternatives to maximum likelihood estimators and method of moment estimators if there's only one observation?

  • 1
    $\begingroup$ Can you tell us what this one data point represents? How informative is it likely to be? More context would be very helpful. $\endgroup$
    – guest
    Apr 7, 2012 at 6:35
  • 2
    $\begingroup$ It sounds like when you say you have "one sample" you mean that the sample you have has only one observation. Is that correct? The terminology can get ambiguous, but usually when someone refers to a sample, they are referring to a collection of observations (usually modeled as realizations from random variables). $\endgroup$
    – cardinal
    Apr 7, 2012 at 13:13
  • $\begingroup$ I think "one observation" is more correct. I've fixed it. $\endgroup$ Apr 7, 2012 at 13:19
  • 2
    $\begingroup$ What do you mean by "work well"? In non-Bayesian perspectives, there is either a decision-theoretic evaluation through a loss function (in which case you cannot avoid being Bayesian by complete class theorems), or an ad hoc restriction on the estimators (like unbiased, minimum variance, &tc.). The choice of the "best" estimator is impossible without a criterion defining "best". $\endgroup$
    – Xi'an
    Apr 7, 2012 at 17:18

1 Answer 1


Are you interested in prediction or inference? If you actually know the distribution (which in practice you never do, except for binary data), there are classical results that show you can't really beat the MLE if your sample size is reasonable. With small sample sizes, penalized likelihoods can do well for prediction, such as the Elastic Net.

Also, you don't have to be a Bayesian in order to use Bayesian methods. All Bayesian methods have frequentist properties (confidence intervals, p-values and the like), but they can be difficult to compute. Frank Samaniego, from ucdavis, has a lot of nice theoretical results on this issue.

  • $\begingroup$ What are some good estimators when I have just 1 sample? (I've also edited the question) $\endgroup$ Apr 7, 2012 at 5:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.