40
$\begingroup$

In statistical inference, problem 9.6b, a "Highest Density Region (HDR)" is mentioned. However, I didn't find the definition of this term in the book.

One similar term is the Highest Posterior Density (HPD). But it doesn't fit in this context, since 9.6b doesn't mention anything about a prior. And in the suggested solution, it only says that "obviously $c(y)$ is a HDR".

Or is the HDR a region containing the mode(s) of a pdf?

What is a Highest Density Region (HDR)?

$\endgroup$
1
  • $\begingroup$ Yes. The amazon page is the book (purchase page). The pdf is the solution to the problems in the book. $\endgroup$ Commented Apr 27, 2015 at 14:17

5 Answers 5

60
$\begingroup$

I recommend Rob Hyndman's 1996 article "Computing and Graphing Highest Density Regions" in The American Statistician. Here is the definition of the HDR, taken from that article:

Let $f(x)$ be the density function of a random variable $X$. Then the $100(1-\alpha)\%$ HDR is the subset $R(f_\alpha)$ of the sample space of $X$ such that $$R(f_\alpha) = \{x\colon f(x)\geq f_\alpha\},$$ where $f_\alpha$ is the largest constant such that $$P\big(X\in R(f_\alpha)\big)\geq 1-\alpha.$$

Figure 1 from that article illustrates the difference between the 75% HDR (so $\alpha=0.25$) and various other 75% Probability Regions for a mixture of two normals ($c_q$ is the $q$-th quantile, $\mu$ the mean and $\sigma$ the standard deviation of the density):

HDR

The idea in one dimension is to take a horizontal line and shift it up (to $y=f_\alpha$) until the area between the $x$ axis and the part of the density where $f(x)\geq f_\alpha$ is $1-\alpha$. (Thanks to bobverity for pointing out my error here - now corrected.) Then the HDR $R_\alpha$ is the projection to the $x$ axis of this area.

Of course, all this works with any density, whether Bayesian posterior or other.

Here is a link to R code, which is the hdrcdepackage (and to the article on JSTOR).

$\endgroup$
3
  • 4
    $\begingroup$ Free link to the paper here $\endgroup$ Commented Aug 31, 2021 at 17:13
  • $\begingroup$ "the area above the 𝑥 axis and under the part..." possibly you might use "the area between the x-axis and the part...". But I am not sure $\endgroup$ Commented Mar 1 at 17:09
  • $\begingroup$ @SextusEmpiricus: that does seem to flow a bit better. Thanks! $\endgroup$ Commented Mar 1 at 17:11
9
$\begingroup$

A highest posterior density [interval] is basically the shortest interval on a posterior density for some given confidence level. A highest density region is probably the same idea applied to any arbitrary density, so not necessarily a posterior distribution.

If $1-\alpha$ is your confidence level, you can always find two quantiles $q_{1-\alpha/2 + c}$, $q_{\alpha/2 -c}$ that will give you a working interval. There are a bunch though, and they all have different lengths. You want the shortest.

If your density $f(\cdot)$ is unimodal, then the shortest interval will happen at the two quantiles $a$ and $b$ such that $f(a) = f(b)$.

$\endgroup$
6
+200
$\begingroup$

I don't have enough reputation to comment, but I think the current top answer by @StephanKolassa may contain a mistake?

"The idea in one dimension is to take a horizontal line and shift it up (to 𝑦=𝑓𝛼) until the area above it and under the density is 1−𝛼. Then the HDR 𝑅𝛼 is the projection to the 𝑥 axis of this area."[emphasis added]

In the figure, this would imply that all the area between the horizontal dotted line and the density curve equals 1-alpha. But I think the text from Rob Hyndman's article implies a different area, which is above the x-axis and under the density for the values in x such that f(x) >= f_a. This would be all the area between the two sets of vertical dotted lines in the figure. Small difference, but threw me for a while when thinking about it.

$\endgroup$
1
  • 2
    $\begingroup$ This is absolutely correct. Thank you for pointing this out! I have corrected my answer and pointed to your answer here. Sorry for not going about this earlier, but I only now saw your answer here now. $\endgroup$ Commented Mar 1 at 16:28
4
$\begingroup$

Hyndman (1996):

  • The region covering the sample space for a given probability 1-α, should have the smallest possible volume.

  • Every point inside the region should have probability density at least as large as every point outside the region.

such regions are called highest density regions (HDR’s)

One of the most distinctive property of HDR’s is that of all possible regions of probability coverage, the HDR has the smallest region possible in the sample space. “Smallest” mean with respect to some simple measure such as the usual Lebesgue measure; in the one-dimensional continuous case that would be the shortest interval, and in the two-dimensional case that would be the smallest area of the surface. In Bayesian analysis a similar approach is called the highest posterior density region (HPD) and the posterior density is used as a measure.

HPD is one of the methods for defining a credible interval in Bayesian statistics.

A credible interval is an interval within which an unobserved parameter value falls with a particular probability. It is an interval in the domain of a posterior probability distribution or a predictive distribution. The generalisation to multivariate problems is the credible region.

Credible intervals are not unique on a posterior distribution. Methods for defining a suitable credible interval include:

  • Choosing the narrowest interval, which for a unimodal distribution will involve choosing those values of highest probability density including the mode (the maximum a posteriori). This is sometimes called the highest posterior density interval (HPDI).
  • Choosing the interval where the probability of being below the interval is as likely as being above it. This interval will include the median. This is sometimes called the equal-tailed interval.
  • Assuming that the mean exists, choosing the interval for which the mean is the central point.
$\endgroup$
0
0
$\begingroup$

This should be the answer for this question:

Highest density regions are often the most appropriate subset to use to summarize a region, and are capable of exposing the most striking features of the data than most alternative methods (Hyndman, 1996)

Hyndman further argues that highest-density regions (HDR) are a “more effective summary of the forecast distribution than other common forecast region” because of its flexibility “to convey both multimodularity and asymmetry in the forecast density”

This should be the answer. Not sure why you start off with a mathematical definition and talk about 100(1−α)%. All I want to know is what is meant by the term HDR (high density region). A classic example of complicating things when not asked :)

$\endgroup$
1
  • $\begingroup$ This argues that HDRs are "often the most appropriate subset", but (a) does not explain why, beyond linking back to the paper, and (b) does not answer the question "What is a Highest Density Region (HDR)?" $\endgroup$ Commented Mar 1 at 16:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.