Calculating “distances” from quantiles

I am trying to compare the measurements from two different methods, A and B, that have each reported 16%, 50%, and 84% quantiles for a measurement X. How can I capture in a number how far away A and B are?

I would like to make some statement like "A is $3\sigma$ away from B" but I doubt it is possible in this case.

16% and 84% quantiles are typically chosen with a mathematical fact in mind: in a normal or Gaussian distribution those quantiles are (closely, not exactly) 1 standard deviation away from the mean.

Hence you have three rough ways of estimating the SD of any variable reported that way:

84% quantile $-$ 50% quantile

50% quantile $-$ 16% quantile

(84% quantile $-$ 16% quantile) / 2

If these estimates are close, you may be good and Gaussian, although there are no guarantees. If these estimates are very different, then watch out.

Note also that you can use these quantiles to get a handle on asymmetry or skewness: in a symmetrical distribution, 16% and 84% quantiles will be equally distant from the 50% quantile, although equal spacing doesn't guarantee that the entire distribution is symmetric. But unequal spacing implies asymmetry, period, although whether it is important depends on magnitudes.

However, your question pivots on comparing samples, each reported in this way, which is more complicated. At best, the two interquantile ranges (and possibly the SDs) will be similar between datasets and then you can relate any difference between to shared variability.

This is mostly cautious or negative, but note a positive: these quantiles can be transformed by any convenient monotonic transformation, as (e.g.) log of quantile $=$ quantile of log, setting aside small print on exactly how the quantiles were calculated, possibly by interpolation between sample values. Hence you may find that there is a transformed scale on which variability of samples is about the same, which is then a scale on which to think.

Counsel of perfection or otherwise: nothing beats a view of all the data and good graphs of the same. Failing that, plotting three summary points for all samples may help you think about the data.

• Thanks for this detailed answer. I am indeed plotting the data, but my advisor wants me to make my plot colorful, so I plan on coloring the points by their rough distances. – rhombidodecahedron Feb 22 '16 at 15:26