4
$\begingroup$

I have calculated a posterior distribution where the highest probability (peak of posterior-curve) is at 99%. But the mean probability is lower, at about 98%. This is of course because "the curve" stretches much further toward 0 than it can toward 100.

I see in literature that the mean value is what matters (and the distribution around it), but my instincts say the peak-point (where the highest probability is) is more relevant.

Is there a conceptual explanation why (in my case) 98% (mean) is a more relevant result than 99% (peak)?

$\endgroup$
  • 1
    $\begingroup$ I think that you need to explain which literature you are looking at in order get a good answer to your question. $\endgroup$ – Dave Apr 19 '13 at 17:14
3
$\begingroup$

I think the frequentist analogues are that of estimating equations to posterior mean and maximum likelihood to posterior mode. They are not equivalent by any means, but have some important similarities. When you estimate a posterior mode, you're doing Bayesian "maximum likelihood".

The posterior mode is not often preferred because the sampling distribution of this value can be very irregular. That's for two reasons: the posterior may have many local maximae and mode estimation is very inefficient except when making strong assumptions. These points are moot when doing exact Bayes, in which case the posterior is known to fall into a parametric family. But doing Gibbs Sampling all higgeldy piggeldy will not guarantee that the posterior falls into any "known" family of distributions.

In basic probability problems, it's easy to obtain exact expressions for posteriors when there are constraining assumptions made about the distribution of sample data and the specification of the prior. In practice, this is rarely the case and posteriors in finite (small) samples can be bumpy, ugly things.

The sampling distribution of the posterior mode does have some convergence properties, like any estimator. But none so well understood and explored as those of the posterior mean. It's so often the efficient estimator in frequentist problems, little wonder it is preferred in the Bayesian world as well.

$\endgroup$
4
$\begingroup$

Both are used (along with the median). Which is "best" depends on the context of how you are going to use it. Generally to a Bayesian the whole posterior distribution is interesting, not just one single number from it. Also interesting is the Credible intervals, but again you have choices, do you want the Highest Posterior Density? or the interval with equal probability in each tail? The HPD gives the narrowest interval, but the second means that when it does not contain the truth it is equally likely to miss on either side. There are other ways to construct the interval as well.

So, How do you plan to use your single number? If it is the answer to a test or homework problem then use whatever your teacher specifies. If you want to gain understanding, then use the entire posterior. If this is for a client then you should start with a CI at a minimum, a single number will likely be misleading.

$\endgroup$
2
$\begingroup$

It's not always the case that the mean is more relevant than the mode. That is part of the value of representing the full distribution within the Bayesian approach, if you have the full distribution you can extract whatever statistical information is required.

The average of the distribution, will often be useful for describing the net (summed or averaged) result of multiple trials due to the central limit theorem; which indicates that the average of a large number of trials converges to the mean of the underlying distribution.

However, in other cases, knowing the mode, or a description of the distribution near the mode, can also be useful, e.g. using the method of steepest descent approximation.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.