3
$\begingroup$

Given a model with a closed-form likelihood, what are reasons to use Bayesian parameter estimation instead of maximum likelihood?

In other words, what are additional estimation results that you can get with Bayesian parameter estimation but not with maximum likelihood estimation?

I am NOT looking for a discussion of the different underlying paradigms but rather something more "tangible". For example, something along the lines of more accurate intervals for predictions or observation-specific parameter estimates for exogenous covariates or handling of missing values or...

$\endgroup$
7
$\begingroup$

I think the big difference is that the Bayesian approach gives you an actual posterior distribution, the likelihood itself is not a distribution. The mode of the posterior can be similar to the maximum likelihood estimate (when the prior does not affect the posterior much), with the posterior you can also interpret the median, mean, and other summaries of the distribution. With the actual posterior you can answer questions like what is the probability that the parameter lies in this range.

If you have co-linearity (multiple regression for example) then the maximum likelihood estimate can be very unstable (things change quite a bit based on including/excluding a single point). Things like ridge regression, lasso, or elasticnet can help stabilize the results, but they also move away from a closed form solution with simple inference. Placing priors on the parameters and doing a Bayesian analysis can give gains equivalent (or superior) to ridge regression or lasso, and still allow inference without being any more complicated.

One example where a Maximum likelihood approach fails, but the Bayesian does not is the Hauck-Donner effect. This happens sometimes with logistic regression or other binomial based models (and could happen in other cases) when a very strong effect looks non-significant because the Wald variance approximation is inflated by the likelihood being fairly flat.

Also, just because a problem has a closed for solution does not mean that it is easy to find or that the computer can do all the calculations in our lifetimes. A Bayesian approach with estimating the posterior with McMC can sometimes be quicker (realistic) compared to finding the exact closed form solution.

And never under estimate the value of being able to incorporate prior information.

Hopefully others will have additional examples.

| cite | improve this answer | |
$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.