2
$\begingroup$

NB: In this post, I will use the abbreviation $TM_P$ to stand for a symmetric truncated (or trimmed) mean that discards the largest and smallest $P/2$ percent of the data. In fact, for concreteness I will refer mostly to $TM_{50}$, but the same question could be asked for any other "similar value" of $P$.


Question

What are situations for which a sound statistical basis exists to prefer $TM_{50}$ (or $TM_P$ for some other more appropriate $P$) over the median as measure of the data's central tendency?


EDIT: (In response to Nick Cox's ridiculing of my wording.) Here's an example of the sort of justification (speaking very broadly) I had hoped for. Q: Why choose the median over the mean? A: For robustness against outliers. The mean can be made anything one wants by a single sufficiently extreme outlier, whereas the median is naturally immune to such an instability; no outlier censoring required. Granted, very extreme outliers are probably entirely tangential to the process under study, but this reasoning still vividly underscores the robustness of the median. There's nothing subjective about it, it's a mathematical fact, even if different people find the same mathematical fact more or less problematic.


Background

I cannot think of a justification for using $TM_{50}$ that does not apply equally well to the median 1. Furthermore, I figure that, as summary statistics go, the median is the conceptually and analytically simpler of the two (and, therefore, the more thoroughly studied and better characterized one). Thus, my gut reaction is to always prefer the median over $TM_{50}$. The motivation for this thread is to either confirm or refute this gut reaction.


1 Of course, the median can be thought of as, roughly, the limit of $TM_P$ as $P \to 100$. Therefore, I figure that any justification for using $TM_P$ over the median can only get weaker as $P \to 100$. An analogous consideration applies when $P \to 0$, if we also replace the median with the mean.

$\endgroup$
  • 2
    $\begingroup$ Reduced variance maybe?! $\endgroup$ – Xi'an Dec 1 '15 at 18:26
  • $\begingroup$ What would a "sound ...basis" look like in this case, especially if it has to overcome your "gut reaction"? (Insert emoticon to taste.) You can get results, if only by simulation, for specified distributions, but the main merit of trimmed means in my view is in the face of genuine uncertainty or insecurity about the generating process. Besides, no one so far as I know is obliged to choose a particular $P$ (in your notation): it makes sense to experiment: see e.g. stats.stackexchange.com/questions/117950/… for flavour. $\endgroup$ – Nick Cox Dec 1 '15 at 18:37
  • $\begingroup$ $TM_{50}$ has a specific name (midmean) and a concrete interpretation as the mean of the values inside the box in a box plot. Neither fact is a justification in principle, naturally, but that may help make them more recognisable to many data analysts. $\endgroup$ – Nick Cox Dec 1 '15 at 18:39
  • $\begingroup$ @NickCox: I am suspicious of experimentation when it is not guided by principle. It too easily degenerates into data-snooping and cherry-picking the most publishable angle. I operate in a world where these are constant threats, so I'm perhaps somewhat paranoid about them. "...the main merit of trimmed means in my view is in the face of genuine uncertainty of insecurity about the generation process": as I said, in this case the median is just as good. $\endgroup$ – kjo Dec 2 '15 at 3:14
  • $\begingroup$ @NickCox: "I operate in a world..." That came out overly dramatic, didn't it. Let me put it differently: I am constantly under a lot of pressure to sign on to statistical procedures that I consider inadequately justified, and that I suspect are chosen only because they were found, by trial-and-error (IOW, experimentation), to present some results in their most favorable light. I am trying to resist this pressure with something more objectively grounded and workable than a suspicion of malpractice. $\endgroup$ – kjo Dec 2 '15 at 4:04
2
$\begingroup$

If you are comparing, say, the sample median, the sample interquartile mean (trimmed mean with 25% of data removed from highest and lowest values) and the sample mean, you have to say what you are trying to do. Otherwise asking which one is better makes no sense at all. If your goal is to estimate the 'center' of a population, you will have to face the problem that these are estimators of DIFFERENT POPULATION PARAMETERS. In this sense, they are not really comparable. For instance, with life expectancies, do you want to estimate the life expectancy that is attained by half of the people? Than you want a median. If you want the average population life expectancy, you'll estimate that with a sample mean. If you want the average life expectancy of the middle 50% of the population, you'll want to estimate it with the interquartile mean. These are not the same value if the population isn't symmetric.

With symmetric populations, all of these are, in some sense, estimators of the same parameter (I say, in some sense, because the sample mean isn't a consistent estimator of population center for some really thick tailed distributions). You'd want to pick the one that minimizes some reasonable loss function (variance?) for the distributions you are likely to work with. Under this criterion, we CAN compare the three.

In general, the mean uses more of the information in the data than does the trimmed mean, which uses more than the median. On the other hand, the median is extremely robust to errors in the data, the trimmed mean somewhat less so and the mean very susceptible to being ruined by outliers.

If your goal is to look for a shift in location for your distribution (for instance, does the life expectancy increase by a fixed amount in one treatment group), you can base comparisons between treatment groups on any measure of center, even if the distribution isn't symmetric. Then you'd select the one that has the lowest variance.

It's pretty easy to compare these via simulations (here I'm using the median, mean and the trimmed mean with 40% trimmed from both extremes). Note that these are all symmetric distributions:

> sims = matrix(ncol=3,nrow=100000,NA)
> colnames(sims) = c("mean","trimmed mean","median")
>  for(i in 1:100000){
+   x = rcauchy(20)
+   sims[i,1] = mean(x)
+   sims[i,2] = mean(x,trim=0.4)
+   sims[i,3] = median(x)
+ }
> #  Variances
> diag(var(sims))
        mean trimmed mean       median 
1.865621e+06 1.360353e-01 1.397239e-01 
> 


>  for(i in 1:100000){
+   x = rnorm(20)
+   sims[i,1] = mean(x)
+   sims[i,2] = mean(x,trim=0.4)
+   sims[i,3] = median(x)
+ }
> #  Variances
> diag(var(sims))
        mean trimmed mean       median 
  0.05007671   0.06883245   0.07347544 
> 


>  for(i in 1:100000){
+   x = rcauchy(20)^3
+   sims[i,1] = mean(x)
+   sims[i,2] = mean(x,trim=0.4)
+   sims[i,3] = median(x)
+ }
> #  Variances
> diag(var(sims))
        mean trimmed mean       median 
2.023236e+27 2.045000e-01 1.308861e-01 
> 

For this simulation:

For the normal distribution, the mean has a higher variance than the others. For the Cauchy, the trimmed mean is best (variance of mean is actually infinite!), while for the cubed Cauchy, the median beats both of them.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.