4
$\begingroup$

Let's consider the interquartile range (IQR), the standard deviation (SD) and the mean absolute deviation (MAD). We know that "one of the most common robust measures of scale is the interquartile range (IQR)", while the "standard deviation, is greatly influenced by outliers", with "a breakdown point of 0". Also, "The mean absolute deviation is a measure of dispersion more robust than the standard deviation".

In cases where I calculate IQR, SD and MAD in approximately symmetric or moderately skewed distributions (with no outliers), SD and MAD give similar values (MAD returns a bit lower values than SD), and they are always lower than IQR values, and relatively far from IQR values.

Probably, nothing is wrong to keep both IQR, SD and MAD, since they are just different definitions of the word "dispersion", but if a person needs to rely on one number indicating the dispersion of the data and asks, "what is the dispersion of that distribution?", should we say all the values of IQR, SD and MAD?

Or should we - for example - discard IQR, since quite far from SD and MAD, and just communicate either SD or MAD, since quite close to each other?

From this case, I would generalise and ask: Are there cases where we need to avoid the usage of IQR?

Improve this question
$\endgroup$
3
  • 2
    $\begingroup$ For a normal distribution $N(\mu, \sigma^2)$ (rather than a sample from a normal distribution), the standard deviation is then $\sigma$, the mean absolute deviation $\sqrt{\frac2\pi}\sigma \approx 0.80 \, \sigma$, the median absolute deviation $\Phi^{-1}\left(\frac34\right)\,\sigma \approx 0.67\, \sigma$, and the interquartile range $2\,\Phi^{-1}\left(\frac34\right)\,\sigma \approx 1.35\, \sigma$. They are all proportional to scale and will usually vary with the shape of the distribution you are looking at. $\endgroup$
    – Henry
    Commented Dec 9, 2023 at 17:14
  • $\begingroup$ "should we say all the values of IQR, SD and MAD" seems to presume these are the only possible values of spread one might consider, but this is not the case. There's a very wide array of choices of measure (infinitely many are possible, though the number that have been used in practice is perhaps only a few dozen). Different choices have different properties; they respond differently to different parts of the distribution. $\endgroup$
    – Glen_b
    Commented Aug 10 at 14:55
  • $\begingroup$ "If a person needs to rely on one number and asks which one...?" Always ask back how exactly the number is going to be used - this may lead to further discussion and you may find out more about how appropriate or inappropriate these number may be in that situation. $\endgroup$
    – Christian Hennig
    Commented Aug 10 at 23:04

2 Answers 2

Reset to default
16
$\begingroup$

First, there's no reason IQR ought to be close to SD or MAD. While they are all measuring some aspect of dispersion, they are doing it in very different ways. An analogy might be that height, width, and depth are all measures of the size of an object, but there's no reason for them to agree. (Not a perfect analogy, I admit).

Second, if a person asks, "what is the dispersion of that distribution?" then we should investigate the distribution and talk to the person about what they want. There is no one right answer. Sometimes, there is no right, or even good, numeric answer and we will need to show a graph or use more than one measure. They may want a single measure, but .... you can't always get what you want, but if you ask a good data analyst, you'll get what you need.

Third, although some people give the IQR as a single number (Q3 - Q1), I prefer to give both numbers. That gives more information.

Finally, we should avoid IQR when it does not answer the needs of the person. This depends only partially on the shape of the distribution. But, e.g. there are cases where Q1, Q2 (median) and Q3 are all the same, but there are a few extreme outliers. There, it seems unlikely that the IQR is an answer to a question that the person could have (but it might be!)

Improve this answer
$\endgroup$
3
  • 4
    $\begingroup$ I love the last sentence of your second paragraph. $\endgroup$
    – Stephan Kolassa
    Commented Dec 8, 2023 at 13:00
  • $\begingroup$ Since the interquartile range (IQR) just refers to the central half of the distribution $-$ contrary to the standard deviation that takes values from the whole distribution $-$, could the usage of IQR be misleading when used for symmetric or approximate symmetric distributions? Indeed, in some books, I read that it is usually recommended to use mean+standard deviation for symmetric distributions and median+interquartile range for non-symmetric distributions. I mean, I understand the recommendation for "median+interquartile range for a non-symmetric dist.", but not that one for symmetric dist. $\endgroup$
    – Ommo
    Commented Dec 13, 2023 at 12:25
  • $\begingroup$ How could it be misleading? The mean and SD have some nice properties when the distribution is symmetric, but that doesn't make the IQR misleading. $\endgroup$
    – Peter Flom
    Commented Dec 13, 2023 at 16:40
5
$\begingroup$

@Peter Flom has given an excellent answer. I want just to gather together a few extra standard comments that deserve some emphasis.

  • Always plot your data. I often find that box plots emphasizing the IQR together with other details -- although much better than no graph at all -- often don't show enough detail or are too easy to misinterpret. If you're comparing say 30 or 100 groups or variables, you need a highly condensed display like parallel box plots. Otherwise for 1 or 2 or 3 or 10 groups or variables, almost any plot that shows all the data is often preferable, even histograms so long as they show enough detail. In my view, no plot design is ideal for all kinds of data, but a quantile-box plot such as those here shows summary box(es) for those who wish and full detail on the individual distribution, including ties.

  • It's just nonsense to advise avoiding the SD just because distributions are asymmetric. The SD is the, or at least a, natural summary of variability (scale, dispersion, spread) for (0, 1) indicators (whose distributions are usually skewed and for which the IQR is usually uninformative); for most binomial distributions; for all Poisson and exponential distributions; and for many other distributions.

  • Conversely, prudence and caution are always in order because SDs can be blown up by outliers, but then we're back to the first point, always look at the data too.

To answer the question itself, the clearest case for avoiding the IQR is when it's zero or otherwise uninformative.

Improve this answer
$\endgroup$
3
  • $\begingroup$ +1 for the second pointer. $\endgroup$
    – User1865345
    Commented Aug 10 at 11:40
  • 1
    $\begingroup$ A couple of random comments: any sample quantile including those on which IQR is based has poor performance when the distribution is not very continuous. Ties in the data make quantiles too insensitive and too unstable simultaneously. Second, be sure to take a look at Gini’s mean difference. $\endgroup$
    – Frank Harrell
    Commented Aug 14 at 14:24
  • $\begingroup$ Thanks @Nick Cox! $\endgroup$
    – Ommo
    Commented Aug 14 at 15:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.