3
$\begingroup$

I've stumbled upon a paper (Eq. 1) that computes a "statistically robust mean" of a distribution using the following formula:

$EM = \frac{1}{2} Q_2 + \frac{1}{4} (Q_1 + Q_3)$

where $Q_{1, 2, 3}$ are first, second and third quartiles of the distribution.

Is there a common name for the formula above?

I can see how it is more robust to outliers than mean, but what are the other properties and potential drawbacks (e.g. when comparing to a simple median?)

$\endgroup$
2
  • 3
    $\begingroup$ The estimator ($EM$) can be thought of as part of a class of L-estimators. L-estimators are linear combinations (weighted sums) of order statistics, the order statistics can be translated into quartiles and medians. However, the median itself is a robust measure of location. I can't really see why the authors used $EM$, but that could be my ignorance. $\endgroup$
    – jcken
    Commented Mar 19, 2021 at 9:04
  • 1
    $\begingroup$ It is not a serious ambiguity but many statistical people seeing EM would think expectation-maximization. $\endgroup$
    – Nick Cox
    Commented Mar 19, 2021 at 11:05

1 Answer 1

3
$\begingroup$

John W. Tukey called this the trimean. (That term is also used in the paper you cite.) It features more in the preliminary editions of Exploratory Data Analysis (three volumes, 1970-1971) than in the fully public 1977 version (both Reading, MA: Addison-Wesley). (More is a little ironic here: I am not sure it is mentioned at all in the 1977 book, but do not have a copy at hand.)

The context for Tukey, his students and his readers around 1970 was identifying median and quartiles by hand in a batch of numbers (usually as individual quantiles or order statistics or averages of adjacent quantiles) and then -- in the case of trimeans -- combining them with weights. Although Tukey was involved with computers from the late 1940s (suggesting the word bit and being an early user of the word software and suggesting hardware tweaks too), his exploratory style around 1970 put very heavy emphasis on mental arithmetic with paper, pens and pencils, both for himself (he did a lot of exploratory work on planes and in committee meetings) and for others. He was keen as far as possible on just ordering, counting and averaging pairs of numbers. The technological context was that, if I recall correctly, cheap portable calculators were not routine until the late 1970s. Use of computers was a long way from the obvious everyday and universal practice it later became.

Similar ad hoc devices were earlier in routine use in sedimentology. From 1957 comes the idea of averaging the 16th, 50th and 84th percentiles of log of particle size. The explicit rationale, such as it is, is that in a Gaussian the 16th and 84th percentiles are about 1 standard deviation away from the mean; the implicit rationale is that such a measure trades off using the information and being resistant to outliers. The technical context was just plotting data on (log)normal probability paper and reading off three data points by interpolation.

So also with the trimean: the major point is to use a bit more information about the distribution than the median provides.

That is mostly history. A detail that may still bite occasionally is that if all you see in a report -- especially of other people's work -- is say median and quartiles, then a trimean combines the information in all three, uses more information than the median alone, and remains more robust than the mean. The weights 1/4, 1/2, 1/4 are arbitrary other than summing to 1, unless someone can suggest a rationale. In Tukey's case cutting down on arithmetic would have been a strong incentive for those weights. There is also an echo of weights (1/4, 1/2, 1/4) for moving averages, which Tukey called Hanning, although Julius von Hann surely didn't invent it.

I'd say there is a continuing rationale for trimmed means, based on ignoring a fraction in each tail and averaging the others. The usual context now is having all the data in memory. Averaging all the values between the quartiles is a 25% trimmed mean, in use since the early 20th century at least and often called a midmean. People familiar with box plots can think of this as averaging all the values inside each box. There is small print about the exact rules, which can differ. I'd tend to use midmeans rather than the trimean, although often they will be very similar. I think it's easier to explain any trimmed mean than the trimean. But seeing how trimmed means might vary with the trimming fraction is a simple tactic, discussed by (but certainly not invented in) this paper.

The paper cited in the original question talks about truncated means. I would say that the term trimmed mean is more common in statistical literature, but that is at most a detail, and they are one and the same.

I will put in a plug for geometric means as well as medians for any distribution that is right-skewed and always positive. The recipe exp(mean(log()) can provide another fair trade-off between using all the information and dampening outliers and/or long tails. They are more or less natural for anything close to lognormal.

EDIT There are other discussions on CV. I've filtered out some mentions that add little, but many helpful details and references are given in

Approximate mean from .25, .5, .75 percentiles

How can I interpret a plot of trimming percentage vs. trimmed mean?

What location parameter is modelled by robust regression?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.