How to calculate the truncated or trimmed mean?

Question

How can I calculate the truncated or trimmed mean? Let's say truncated by 10%?

I can imagine how to do it if you have 10 entries or so, but how can I do it for a lot of entries?

Answer 1

Trimmed mean involves trimming $P$ percent observations from both ends.

E.g.: If you are asked to compute a 10% trimmed mean, $P = 10$.

Given a bunch of observations, $X_i$:

1. First find $n$ = number of observations.
2. Reorder them as "order statistics" $X_i$ from the smallest to the largest.
3. Find lower case $p = P/100$ = proportion trimmed.
4. Compute $n p$.

If $n p$ is an integer use $k = n p$ and trim $k$ observations at both ends.

$R$ = remaining observations = $n - 2k$.

Trimmed mean = $(1/R) \left( X_{k+1} + X_{k+2} + \ldots + X_{n-k} \right).$

Example: Find 10% trimmed mean of

2, 4, 6, 7, 11, 21, 81, 90, 105, 121

Here, $n = 10, p = 0.10, k = n p = 1$ which is an integer so trim exactly one observation at each end, since $k = 1$. Thus trim off 2 and 121. We are left with $R = n - 2k = 10 - 2 = 8$ observations.

10% trimmed mean= (1/8) * (4 + 6 + 7 + 11 + 21 + 81 + 90 + 105) = 40.625

If $ n p$ has a fractional part present, trimmed mean is a bit more complicated. In the above example, if we wanted 15% trimmed mean, $P = 15, p = 0.15, n = 10, k = n p = 1.5$. This has integer part 1 and fractional part 0.5 is present. $R = n - 2k = 10 - 2 * 1.5 = 10 - 3 = 7$. Thus $R = 7$ observations are retained.

Addendum upon @whuber's comment: To remain unbiased (after removing 2 and 121), it seems we must remove half of the 4 and half of the 105 for a trimmed mean of $(4/2 + 6 + 7 + 11 + 21 + 81 + 90 + 105/2)/7 = 38.64$

Source: Class notes on P percent trimmed mean

Answer 2

In addition to the above answer, if there are many entries (say n), then first sorting them takes time O(n log n). However, there is a linear-time solution.

1. Compute the P-quantile L and (1-P)-quantile U. There is a simple (quicksort-like) algorithm for this that runs in expected linear time. There is also a more complicated algorithm that runs in worst case linear time. Both can be found, for example, in: Cormen, Leiserson, Rivest, Stein: Introduction to Algortithms.

2. Scan through all values and add those between L and U. This obviously takes linear time.

3. If there are ties and the computed quantiles exist several times among the values, we might have added too many or too few values and may need to correct for this appropriately. Since we know how many numbers we added in step 2, and also how many times we have seen L and U, this can be done in constant time.

4. Divide the total sum by the number of summands.

Note that the above recipe is only worthwhile if n is really large and sorting all of them would be a performance hit, perhaps a few millions.