# What is “the shortest half of the data”?

Here is a histogram (realized with JMP) displaying two types of box plot called outlier box plot and quantile box plot.

Right below, there are a bunch of explanations of the meaning of the different features of the outlier box plot.

The red bracket is "the shortest half of the data (the densest region)". What does this expression exactly mean? How different is it from the quartiles?

Source

• This is the so-called "Shorth" of the data. For perfectly symmetric distributions, it corresponds to the box of the boxplot. – Michael M Sep 11 '14 at 8:14
• – Nick Cox Sep 16 '14 at 18:00
• @MichaelMayer That's true for perfectly symmetric unimodal distributions and some others. A U-shaped symmetric distribution would have either half of the distribution tying for shortest half, but the central box would be longer. – Nick Cox Sep 16 '14 at 18:03
• @NickCox: You are right! Still, the statistic in the OP is called "shorth" in the literature ;). – Michael M Sep 16 '14 at 18:51

## 3 Answers

Think of all the ways that you could split the data into 2 "halves", You could start with the minimum to the median, then go from the 2nd lowest point to the point just above the median, then 3rd lowest point to ... then from the median to the highest point. Measure the length of all the "halves" that you computed and the red bracket shows the one that has the smallest range.

• Do those halves need to be contiguous? I think the answer is yes, but want to be sure. – Alexis Dec 6 '14 at 16:32

The red bracket represents the smallest interval that encompass 50% of the data.

The two quartiles represents the first 25% of the data from each side of the mean encompassing for 50% of the data but this is not necessarily the densest region.

Imagine a skewed distribution that has a very long tail on the right. The median might be much higher than the mode and be fairly high. In consequence the 25% quartile will eventually include the mode but the right quartile will be long in a very sparse space. The densest region necessarily include the median (exception in @whuber's comment below) and one of the two quartile.

• (+1) One of your remarks is somewhat misleading, because the densest 50% region necessarily includes a median. The only way "the" median could not be included is when there are an even number of data values and either the upper or lower half is the densest 50%. But this occurs only because "the" median is conventionally defined as midway between the two middle values. So perhaps a more accurate and useful conclusion would be that a densest 50% region of a dataset always includes a median of the data. – whuber Sep 10 '14 at 16:19

The JMP documentation is a little unclear--it just cites a book: Robust Regression and Outlier Detection.

However, it looks like the bracket contains the "highest density interval", or "highest probability density" interval. This is the smallest (i.e., shortest) interval that contains 50% of the data points.

Suppose your data looks like this

-30, -20, -10, 1, 2, 3, 4, 5, 6, 10, 20, 30.

The 50% densest region runs from 0 to 6 because it contains half the data (six of the twelve points) and is the shortest interval that does so (it's five units long;the next closest (2-10) is eight units long instead).

The highest probability interval often shows up in Bayesian settings, sometimes as an alternative to confidence intervals. Here, they probably mean for you to use it a heuristic for detecting outliers.