Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
In statistical inference, problem 9.6b, a "Highest Density Region (HDR)" is mentioned. However, I didn't find the definition of this term in the book.
One similar term is the Highest Posterior Density (HPD). But it doesn't fit in this context, since 9.6b doesn't mention anything about a prior. And in the suggested solution, it only says that "obviously $c(y)$ is a HDR".
Or is the HDR a region containing the mode(s) of a pdf?
What is a Highest Density Region (HDR)?
I recommend Rob Hyndman's 1996 article "Computing and Graphing Highest Density Regions" in The American Statistician. Here is the definition of the HDR, taken from that article:
Let $f(x)$ be the density function of a random variable $X$. Then the $100(1-\alpha)\%$ HDR is the subset $R(f_\alpha)$ of the sample space of $X$ such that $$R(f_\alpha) = \{x\colon f(x)\geq f_\alpha\},$$ where $f_\alpha$ is the largest constant such that $$P\big(X\in R(f_\alpha)\big)\geq 1-\alpha.$$
Figure 1 from that article illustrates the difference between the 75% HDR (so $\alpha=0.25$) and various other 75% Probability Regions for a mixture of two normals ($c_q$ is the $q$-th quantile, $\mu$ the mean and $\sigma$ the standard deviation of the density):
The idea in one dimension is to take a horizontal line and shift it up (to $y=f_\alpha$) until the area above it and under the density is $1-\alpha$. Then the HDR $R_\alpha$ is the projection to the $x$ axis of this area.
Of course, all this works with any density, whether Bayesian posterior or other.
Here is a link to R code, which is the hdrcdepackage (and to the article on JSTOR).
A highest posterior density [interval] is basically the shortest interval on a posterior density for some given confidence level. A highest density region is probably the same idea applied to any arbitrary density, so not necessarily a posterior distribution.
If $1-\alpha$ is your confidence level, you can always find two quantiles $q_{1-\alpha/2 + c}$, $q_{\alpha/2 -c}$ that will give you a working interval. There are a bunch though, and they all have different lengths. You want the shortest.
If your density $f(\cdot)$ is unimodal, then the shortest interval will happen at the two quantiles $a$ and $b$ such that $f(a) = f(b)$.
Hyndman (1996):
The region covering the sample space for a given probability 1-α, should have the smallest possible volume.
Every point inside the region should have probability density at least as large as every point outside the region.
such regions are called highest density regions (HDR’s)
One of the most distinctive property of HDR’s is that of all possible regions of probability coverage, the HDR has the smallest region possible in the sample space. “Smallest” mean with respect to some simple measure such as the usual Lebesgue measure; in the one-dimensional continuous case that would be the shortest interval, and in the two-dimensional case that would be the smallest area of the surface. In Bayesian analysis a similar approach is called the highest posterior density region (HPD) and the posterior density is used as a measure.
HPD is one of the methods for defining a credible interval in Bayesian statistics.
A credible interval is an interval within which an unobserved parameter value falls with a particular probability. It is an interval in the domain of a posterior probability distribution or a predictive distribution. The generalisation to multivariate problems is the credible region.
Credible intervals are not unique on a posterior distribution. Methods for defining a suitable credible interval include:
