Highest probability set and density ratios equal to probability ratios I came across a pretty result I had not seen before, and wondered if there were more examples


*

*For a random variable with an exponential distribution, if you want the highest probability set to contain all but $p$ of the probability, then you want the density of the values you exclude to be less than $p$ times the density at the mode, and this is easy to show. 


*

*For example if $p=0.05$ and the exponential distribution has parameter $1$, 

*then you want to exclude values with $x > -\log_e\left(0.05\right) \approx 3$ 

*and all their densities are below $0.05$, while the density at the mode at $0$ is $1$; 

*the excluded set covers $5\%$ of the probability. No surprises here.    


*For a bivariate normally distributed random variable, if you want the highest probability set to contain all but $p$ of the probability, then again you want the density of the values you exclude to be less than $p$ times the density at the mode. This is true even if the two parts of the bivariate distribution are correlated.  This seems less obvious and is the pretty result I noticed; it can be proved as a consequence of the earlier exponential example and the relationship between the exponential distribution and the chi-squared distribution with $2$ degrees of freedom.  This normal distribution result does not apply in $1$ dimension or in more than $2$ dimensions.


*

*For example if $p=0.05$ and you have a standard uncorrelated bivariate normal with mean and mode $\mathbb \mu = {0 \choose 0}$ and covariance matrix $\mathbb \Sigma = {1 \: 0 \choose 0 \: 1}$, 

*then you want to exclude values where $\|\mathbf x\| > \sqrt{-2 \log_e\left(0.05\right)} \approx 2.45$ 

*and all their densities are below $\frac{1}{40\pi}$, which is $0.05$ times the density at the mode of $\frac{1}{2\pi}$; 

*again the excluded set covers $5\%$ of the probability. 



Are there other distributions with this property? 
It would not particularly surprise me if there were artificial examples in higher dimensions, or simpler examples for single values of $p$, but are there any simple general examples like the two above?
 A: I am not aware of any other distributions that have this property, though it would certainly be possible to construct distributions of that kind.  Nevertheless, what I can offer is to give some structure to the problem by presenting the relevant functions. 
 Following standard notation in the field, I will use $\alpha$ in place of your $p$, so that you are referring to a HDR with coverage probability $1-\alpha$.

Consider a continuous distribution with density function $f_X$ having supremum $m_X \equiv \sup_x f_X(x)$.  We can define the function $H:[0,1] \rightarrow [0,1]$ via the equation:
$$1-\alpha = \mathbb{P}(f_X(X) \geqslant m_X \cdot H(\alpha)).$$
This does not uniquely define $H$, so it is usual to further stipulate that $H(\alpha)$ is the infimum of the set of all values for which the above equation is satisfied.  If $f_X$ has no flat regions in its support then $H$ will also be continuous, but if there are flat regions in the support of the density the $H$ may have "jumps".  Any any case, it is simple to show that $H$ is a non-decreasing function with $H(0) = 0$ and $H(1) = 1$.
For a HDR with coverage probability $1-\alpha$, suppose we let $r(\alpha)$ denote the cut-off point for the density (i.e., the HDR includes all points with $f_X(x) \geqslant r(\alpha)$).  Then we can write our function of interest as:
$$H(\alpha) = \frac{r(\alpha)}{m_X}.$$
It is possible to derive this function for a number of families of distributions, though in some cases the function will only be implicitly defined.  The condition of interest to you is when $H$ is the identity function (i.e., when $H(\alpha) = \alpha$).  Below we will derive this function for various distributions, to show some distributions that have the property of interest to you, and some distributions that do not have this property.

The exponential distribution: Suppose that $f_X(x) = \text{Exp}(x|\theta)$ where $\theta$ is the rate parameter.  Using the coverage probability $1-\alpha$ the HDR and density cut-off are:
$$\text{HDR}(1-\alpha) = \Bigg[ 0, \frac{|\ln(\alpha)|}{\theta} \Bigg]
\quad \quad \quad r(\alpha) = \theta \cdot \alpha.$$
Since $m_X = \theta$ we therefore have:
$$H(\alpha) = \frac{r(\alpha)}{m_X} = \frac{\theta \cdot \alpha}{\theta} = \alpha.$$
This confirms your observation that the exponential distribution has the property of interest.  As you have pointed out in the comments, it also applies to the chi-squared distribution with two degrees-of-freedom, since that is an exponential distribution with $\theta=\tfrac{1}{2}$.

The normal distribution: Suppose that$f_X(x) = \text{N}(x|\mu, \sigma^2)$ where $\mu$ is the mean parameter and $\sigma$ is the standard deviation parameter.  Using the coverage probability $1-\alpha$ the HDR and density cut-off are:
$$\text{HDR}(1-\alpha) 
= \Bigg[ \mu - z_{\alpha/2} \sigma, \ \mu + z_{\alpha/2} \sigma \Bigg]
\quad \quad \quad r(\alpha) = \frac{1}{\sqrt{2 \pi \sigma^2}} \cdot \exp \bigg( -\frac{z_{\alpha/2}^2}{2} \bigg).$$
Since $m_X = 1/\sqrt{2 \pi \sigma^2}$ we therefore have:
$$H(\alpha) = \frac{r(\alpha)}{m_X} = \exp \bigg( -\frac{z_{\alpha/2}^2}{2} \bigg).$$

