4
$\begingroup$

Let us suppose that I have data that is normally distributed. I wish to find the $\mu$ and $\sigma$ parameters. The common sense thing to do is to simple calculate the sample mean and deviation. No fancy math, be done with it, and understand that a margin-of-error is expected. If we wish to quantify the margin of error we can use a confidence interval.

However, the fancy Bayesian approach is instead provide probability distributions for the two parameters. What benefit does this bring? I imagine there are problems were Bayesian approaches are better, but in this simplistic example that was mentioned what is the benefit?

Improve this question
$\endgroup$
2

1 Answer 1

Reset to default
8
$\begingroup$

One obvious benefit is that given a probability distribution and an evaluation* loss function, you can derive the optimal point estimate, i.e. one that minimizes the expected loss. This is not strictly specific to Bayesian methods but to any method that delivers a probability distribution instead of a point or an interval estimate. However, Bayesian methods are perhaps the most popular among such methods (another option would be fiducial methods).

Another benefit is the ability to seamlessly incorporate useful prior information into the estimate. If you have (strong) prior information, your Bayesian estimate will frequently be more accurate than, say, a frequentist estimate.

A third benefit concerns interpretation. Bayesian methods tell us something about the unknown parameter conditional on the observed data (something that is often of direct interest) rather than about the observed data conditional on the unknown parameter (often not directly relevant). The interpretation of a Bayesian result is thus often more straightforward.

*See here for details of what evaluation loss means and how it differs from estimation loss.

Improve this answer
$\endgroup$
6
  • 1
    $\begingroup$ (+1) nice concise explanations !! $\endgroup$
    – Joe King
    Oct 11, 2021 at 13:04
  • $\begingroup$ @RichardHardy Since asking my question, and after learning more, I have follow-up questions based on your answer. 1) You can "derive optimal point estimate" using MLE, that is an entirely non-Bayesian method, so I do not see what is special to the Bayesian approach. 2) In my experience, with some toy examples with randomly generated data, once you have a lot of data, it really does not matter so much how good your prior is, your estimates are all roughly the same. There is no big benefit to be gained in accuracy. $\endgroup$
    – Nicolas Bourbaki
    Sep 20, 2022 at 20:52
  • 1
    $\begingroup$ @NicolasBourbaki, re 2) there certainly are situations when the prior does not make a big difference, but there are also situations where it does. The Bayesian approach is useful in the latter situations. Re 1) MLE does not care about the loss function used for evaluation; see my answer to this question for details and terminology. The point estimate that MLE delivers may be optimal under a restricted class of loss functions but not under the evaluation loss function that is relevant for you as a particular user. $\endgroup$
    – Richard Hardy
    Sep 21, 2022 at 7:58
  • 1
    $\begingroup$ @NicolasBourbaki, e.g. the MLE of $\mu$ in a Normal($\mu$,1) distribution will not work very well in terms of expected evaluation loss if your evaluation loss function is highly asymmetric (such as a quantile loss function for a 99% quantile). A different point estimate would optimize the expected loss in that case. The Bayesian approach readily accommodates any evaluation loss function, and that is its benefit. $\endgroup$
    – Richard Hardy
    Sep 21, 2022 at 8:00
  • $\begingroup$ @RichardHardy Thank you for your answer, it seems highly relevant to my question, unfortunately, I cannot understand the vocabulary you are using. Would it be possible to write an answer by using an example with simulated fake data. Suppose our data is rbinom(3,1), and we wish to estimate mu using a "quantile loss function". How does the MLE method break down exactly and how does it compare with a Bayesian approach? $\endgroup$
    – Nicolas Bourbaki
    Sep 21, 2022 at 13:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.