8
$\begingroup$

How can I calculate the truncated or trimmed mean? Let's say truncated by 10%?

I can imagine how to do it if you have 10 entries or so, but how can I do it for a lot of entries?

$\endgroup$
  • 1
    $\begingroup$ Should this be tagged trimmed instead of truncated? $\endgroup$ – user28 Nov 5 '10 at 23:57
  • $\begingroup$ I'd say either en.wikipedia.org/wiki/Truncated_mean will do. $\endgroup$ – Queops Nov 6 '10 at 19:11
19
$\begingroup$

Trimmed mean involves trimming $P$ percent observations from both ends.

E.g.: If you are asked to compute a 10% trimmed mean, $P = 10$.

Given a bunch of observations, $X_i$:

  1. First find $n$ = number of observations.
  2. Reorder them as "order statistics" $X_i$ from the smallest to the largest.
  3. Find lower case $p = P/100$ = proportion trimmed.
  4. Compute $n p$.

If $n p$ is an integer use $k = n p$ and trim $k$ observations at both ends.

$R$ = remaining observations = $n - 2k$.

Trimmed mean = $(1/R) \left( X_{k+1} + X_{k+2} + \ldots + X_{n-k} \right).$

Example: Find 10% trimmed mean of

2, 4, 6, 7, 11, 21, 81, 90, 105, 121

Here, $n = 10, p = 0.10, k = n p = 1$ which is an integer so trim exactly one observation at each end, since $k = 1$. Thus trim off 2 and 121. We are left with $R = n - 2k = 10 - 2 = 8$ observations.

10% trimmed mean= (1/8) * (4 + 6 + 7 + 11 + 21 + 81 + 90 + 105) = 40.625

If $ n p$ has a fractional part present, trimmed mean is a bit more complicated. In the above example, if we wanted 15% trimmed mean, $P = 15, p = 0.15, n = 10, k = n p = 1.5$. This has integer part 1 and fractional part 0.5 is present. $R = n - 2k = 10 - 2 * 1.5 = 10 - 3 = 7$. Thus $R = 7$ observations are retained.

Addendum upon @whuber's comment: To remain unbiased (after removing 2 and 121), it seems we must remove half of the 4 and half of the 105 for a trimmed mean of $(4/2 + 6 + 7 + 11 + 21 + 81 + 90 + 105/2)/7 = 38.64$

Source: Class notes on P percent trimmed mean

$\endgroup$
  • $\begingroup$ @Mehper In the last example, which three observations would you remove? Obviously the 2 and the 121, but what else? To remain unbiased, it seems you must remove half of the 4 and half of the 105 for a trimmed mean of (4/2 + 6 + 7 + 11 + 21 + 81 + 90 + 105/2)/7 = 34.64 $\endgroup$ – whuber Nov 5 '10 at 19:59
  • $\begingroup$ @Mehper: just FYI, you can format maths by writing TeX expression in between $ signs. E.g. $X_i$ $\endgroup$ – nico Nov 5 '10 at 20:33
  • $\begingroup$ @whuber: Thanks for your comment, I've added your comment to the answer; @nico: Thanks for letting me know about TeX formatting. I tried to update the answer using TeX format but I couldn't manage it well. Could you please give me a link which explains how to use TeX style in posts? I have no experience in TeX. $\endgroup$ – Mehper C. Palavuzlar Nov 5 '10 at 21:14
  • 1
    $\begingroup$ @Mehper: Google "TeX Manual" and take your pick. I like the "gentle introduction" because it contains useful, readable tables: tex.ac.uk/tex-archive/info/gentle/gentle.pdf $\endgroup$ – whuber Nov 5 '10 at 21:25
  • 1
    $\begingroup$ @Mepher: sure, here you go! mathjax.org/help/user (note that if you right click on any math formula you'll have a context menu linking to that page). You can also use MathML instead of TeX (if you're very brave :P). $\endgroup$ – nico Nov 5 '10 at 21:32
5
$\begingroup$

In addition to the above answer, if there are many entries (say n), then first sorting them takes time O(n log n). However, there is a linear-time solution.

  1. Compute the P-quantile L and (1-P)-quantile U. There is a simple (quicksort-like) algorithm for this that runs in expected linear time. There is also a more complicated algorithm that runs in worst case linear time. Both can be found, for example, in: Cormen, Leiserson, Rivest, Stein: Introduction to Algortithms.

  2. Scan through all values and add those between L and U. This obviously takes linear time.

  3. If there are ties and the computed quantiles exist several times among the values, we might have added too many or too few values and may need to correct for this appropriately. Since we know how many numbers we added in step 2, and also how many times we have seen L and U, this can be done in constant time.

  4. Divide the total sum by the number of summands.

Note that the above recipe is only worthwhile if n is really large and sorting all of them would be a performance hit, perhaps a few millions.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.