How to uniquely define the credible interval?

Question

Consider we are in the Bayesian paradigm and consider that we know the posterior $P(\tau|d)$ of the single parameter $\tau$, whose max of the posterior is given by $\tau_{\text{best}}$.

The credible interval $[\tau_{\text{min}}, \tau_{\text{max}}]$ is defined by defining the indicador function

$$I(\tau) = \begin{cases} 1 & \text{if } \tau_{\text{min}} < \tau < \tau_{\text{max}} \\ 0 & \text{else} \end{cases} $$

and solve $P(I|d) = 0.95$, right?

My issue is that this does not uniquely define the interval since there is only one condition to compute two independent variables (i.e. the system is underdetermined).

Given this, I tried defining it using two conditions

$$P(\tau_{\text{min}} < \tau < \tau_{\text{best}}) = 0.95/2$$ $$P(\tau_{\text{best}} < \tau < \tau_{\text{max}}) = 0.95/2$$

which is to say that I want the mass "0.95" equally distributed on both sides of $\tau_{\text{best}}$.

However, I'm working on an example on which 0.95/2 is not on the left side of the distribution (i.e. the distribution is not symmetrical).

So, my question is: how do I define a unique credible interval that is generic? Is it even possible to define it generically and uniquely?

Answer 1

There are some issues with your wording. I assume when you say $\tau_\text{best}$ you are referring to the Bayes estimator (posterior mean) or perhaps the posterior mode. Also, the min max subscripts would best be defined by lower and upper limits of the credible set.

Regardless, what you want is called the highest posterior density interval (HPDI) which is unique under most conditions. This posting should answer your question.

The R package "coda" contains an algorithm to compute the HPDI from a sample. Also, SAS will compute this interval as part of any Bayesian analysis.

Answer 2

Nathan L is right that the HPDI is one reasonable answer to this question. But it seems worth pointing out that there is no single perfect answer to your question. The posterior is a probability distribution, and there is no perfect way to summarize all the information in a generic probability distribution with just two scalars. One reasonable way to describe a distribution using an interval is to give the HPDI, one way is the 5%-95% interval, and there are other possibilities. None is perfect.

But one nice feature of Bayesian statistics is that the problem of giving a plausible interval for a parameter is divided into two parts, one of which generally has a unique answer, and one of which does not. The unique answer is the posterior, which specifies everything you know about the parameter after you've taken the data into account. If you want to summarize the posterior with an interval, that's fine, but (as I hope is clear now) there's no one true answer to that part of the problem.