4
$\begingroup$

Given a one-dimensional posterior distribution it is often the case that you want to calculate a point estimate and a credible interval for the corresponding parameter. There are, of course, many ways of doing this, but there seem to be a couple of methods that are most often used in practice such as modes with highest posterior intervals and medians with equi-tailed intervals. There are however other proposed methods (for example, Snelson and Ghahramani (2005) and Druilhet and Marin (2007) ).

Is seems like "different methods for constructing posterior point/interval estimates" would be a perfect topic for a review paper/book and my question is: Does such a review exist?

(I've often seen questions regarding "what type of credible interval to use?" where the unhelpful answer is something like: "It depends on your loss function". So my question could equally well be: Does it exist a review of different loss functions for constructing posterior point/interval estimates?)

References

Snelson, E., & Ghahramani, Z. (2005). Compact approximations to Bayesian predictive distributions. In Proceedings of the 22nd international conference on Machine learning (pp. 840-847). ACM.

Druilhet, P., & Marin, J. M. (2007). Invariant HPD credible sets and MAP estimators. Bayesian Analysis', 2(4), 681-691.

$\endgroup$

1 Answer 1

1
$\begingroup$

This is mostly answered by decision theory. The approach is that you must formalize what you want to achieve with your point estimate, and that can be formalized as a loss function. Examples are squared error loss $$ L(\hat{\theta}, \theta) = (\hat{\theta} - \theta)^2, $$ absolute value loss, or others. Then the optimal estimator (in your Bayesian setting) is given by the function $\hat{\theta}$ that minimized posterior expected loss $$ E L(\hat{\theta}, \theta) $$ where the expectation is over the posterior distribution of $\theta$. In the case of the squared error loss, that is the posterior expected value.

You can read about this an any text about Bayesian decision theory, such as Christian Robert: "The Bayesian choice" (Springer).

EDIT

As for the question/commentary in the comment: The loss function is supposed to give your loss structure, and no paper can tell you that! Of course, if yours is an inference problem, so there is no immediate economical or other loss to you, the answer is not easy, but the book I referred to above will contain some discussion. As for the blog you referred, http://doingbayesiandataanalysis.blogspot.se/2012/04/why-to-use-highest-density-intervals.html that is an interestingt point. It advises for highest posterior density intervals (HPI). As far as I understand, that will not follow as conclusion from any decision-theoretic approach. HPI-intervals are not invariant under reparametrization, any proper Bayesian solution must be invariant. The meaning of this is that if your parameter is $\theta$, and you reparametrize the model with $g(\theta)$, where $g$ is some invertible function, then you get the same result directly doing inference for $\theta$, or by first reformulating everything in terms of $g(\theta)$, doing inference on that new scale, and then using $g^{-1}$ to translate that inference back to the original scale $\theta$. That will not work out for HPI, so is not given by any loss function.

$\endgroup$
1
  • 1
    $\begingroup$ "It depends on your loss function"... So, my question was whether a review article of this exists, discussing different, useful, loss functions. I'm looking for more discussion, along the lines of, for example, this blog post: doingbayesiandataanalysis.blogspot.se/2012/04/… $\endgroup$ Commented Oct 2, 2014 at 19:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.