# If mean is so sensitive, why use it in the first place?

It is a known fact that median is resistant to outliers. If that is the case, when and why would we use the mean in the first place?

One thing I can think of perhaps is to understand the presence of outliers i.e. if the median is far from the mean, then the distribution is skewed and perhaps the data needs to be examined to decide what is to be done with the outliers. Are there any other uses?

-
Regarding the first question a quick side note: Mean in statistics is just the first moment of a population, while median is not. Seeking to use CLT, law of large numbers, etc. you are again linked to the existence of finite moments. Though taking for example Cauchy distribution: median exists, while mean doesn't ;) –  Dmitrij Celov Aug 13 '11 at 13:12
@Dmitrij That is a deep and insightful answer. Why don't you elaborate on it in a reply? –  whuber Aug 13 '11 at 14:36
If you didn't use the mean you'd hurt its feelings? (Sorry, couldn't resist.) –  Daniel R Hicks Aug 15 '11 at 1:21
@Daniel R Hicks: And that's quite mean, is it? (Sorry, couldn't resist as well). –  Muhammad Alkarouri Aug 15 '11 at 10:08
This question is much more interesting than the usual, "How come we don't just always use robust algorithms?" question, but may have the same underlying thinking that "robust == magical" and if we just used robust methods, we wouldn't have to examine our data, understand it, or worry about different kinds of accuracy issues, since they're "robust". Still, +1. –  Wayne Aug 15 '11 at 17:48

In a sense, the mean is used because it is sensitive to the data. If the distribution happens to be symmetric and the tails are about like the normal distribution, the mean is a very efficient summary of central tendency. The median, while being robust and well-defined for any continuous distribution, is only $\frac{2}{\pi}$ as efficient as the mean if the data happened to come from a normal distribution. It is this relative inefficiency of the median that keeps us from using it even more than we do. The relative inefficiency translates into a minor absolute inefficiency as the sample size gets large, so for large $n$ we can be more guilt-free about using the median.

It is interesting to note that for a measure of variation (spread, dispersion), there is a very robust estimator that is 0.98 as efficient as the standard deviation, namely Gini's mean difference. This is the mean absolute difference between any two observations. [You have to multiply the sample standard deviation by a constant to estimate the same quantity estimated by Gini's mean difference.] An efficient measure of central tendency is the Hodges-Lehmann estimator, i.e., the median of all pairwise means. We would use it more if its interpretation were simpler.

-
+1 for mentioning Hodges-Lehmann estimator of central tendency. In many respects it is in-between of mean and median. If only it were easy to calculate in large sample it'd be more popular than mean or median as a measure of location, I think. –  ttnphns Aug 13 '11 at 17:07
BTW, @Frank, do you know which theoretical sampling distribution Hodges-Lehmann centre follows? I don't - and I take interest. –  ttnphns Aug 13 '11 at 17:25
Thanks for the comment. A one-liner in R can compute it efficiently up to N=5000: w <- outer(x, x, '+'); median(w[row(w) >= col(w)])/2. A trivial C, Fortran, or Ratfor program could be called by R to make it blazing fast. The ICSNP package in R has a fairly efficient implementation with its hl.loc function. For N=5000 it was 2.66 times faster than the above code (total time 1.5 sec.). It would be nice to also get an confidence interval efficiently. –  Frank Harrell Aug 13 '11 at 17:29

When a value is garbage for us we call it "outliar" and want analysis be robust to it (and prefer median); when that same value is attractive we call it "extreme" and want analysis be sensitive to it (and prefer mean). Dialectics...

Mean reacts equally to a shift of value irrespective to where in the distribution the shift takes place. For example, in 1 2 3 4 5 you may increase any value by 2 - the increase of mean will be the same. Median's reaction is less "consistent": add 2 to data points 4 or 5, and median won't increase; but add 2 to point 2 - so that the shift is over the median, and the median changes dramatically (greatly than mean will change).

Mean is always exactly located. Median is not; for example, in set 1 2 3 4 any value between 2 and 3 can be called median. Thus, analyses based on medians are not always unique solution.

Mean is a locus of minimal sum-of-squared-deviations. Many optimization tasks based on linear algebra (including famous OLS regression) minimize this squared error and therefore imply concept of mean. Median a locus of minimal sum-of-absolute-deviations. Optimization techniques to minimize such error are non-linear and are more complex / poorly known.

-
+1 I have a little concern that the first paragraph might be misunderstood as implying outlier detection is entirely a subjective process. I don't think you mean to imply that, though. –  whuber Aug 13 '11 at 14:33
+1 | I think the first sentence implies that the application of outlier detection is entirely subjective and therefore I vote for keep as is. –  John Aug 13 '11 at 15:18
I meant that outliar detection is stringent procedure with subjective philosophical or moral roots –  ttnphns Aug 13 '11 at 16:59
Quizz: "Optimization techniques to minimize such error are non-linear and are more complex / poorly known." In what decade was this statement last true? –  user603 Aug 14 '11 at 15:00
@ttnphns, the spelling "outliar" instead of "outlier" is intentional, or not? –  mpiktas Aug 16 '11 at 7:28

Lots of great answers already, but, taking a step back and getting a little more basic, I'd say it's because the answer you get depends on the question you ask. The mean and median answer different questions - sometimes one is appropriate, sometimes the other.

It's simple to say that the median should be used when there are outliers, or for skewed distributions, or whatever. But that's not always the case. Take income - nearly always reported with median, and usually that's right. But if you are looking at the spending power of a whole community, it may not be right. And in some cases, even the mode might be best (esp. if the data are grouped).

-
+1 for the obvious point that nobody else seemed to address: they are different concepts and answer different questions. Also in many instances much is lost by condensing the entire distribution into one summary number, so sometimes they both do a lousy job. –  Michael McGowan Aug 15 '11 at 18:38

There are a lot of answers to this question. Here's one that you probably won't see elsewhere so I'm including it here because I believe it's pertinent to the topic. People often believe that because the median is considered a robust measure with respect to outliers that it's also robust to most everything. In fact, it's also considered robust to bias in skewed distributions. These two robust properties of the median are often taught together. One might note that underlying skewed distributions also tend to generate small samples that look like they have outliers and conventional wisdom is that one use medians in such situations.

#function to generate random values from a skewed distribution
rexg <- function (n, m, sig, tau) {
rexp(n, rate = 1/tau) + rnorm(n, mean = m, sd = sig)
}


(just a demonstration that this is skewed and the basic shape)

hist(rexg(1e4, 0, 1, 1))


Now, let's see what happens if we sample from this distribution various sample sizes and calculate median and mean to see what the differences between them are.

#generate values with various n's
N <- 1e4
ns <- 2:30
y <- sapply(ns, function(x) mean(apply(matrix(rexg(x*N, 0, 1, 1), ncol = N), 2, median)))
plot(ns,y, type = 'l', ylim = c(0.85, 1.03), col = 'red')
y <- sapply(ns, function(x) mean(colMeans(matrix(rexg(x*N, 0, 1, 1), ncol = N))))
lines(ns,y)


As can be seen from the above plot the median (in red) is much more sensitive to the n than the mean. This is contrary to some conventional wisdom regarding using medians with low ns, especially if the distribution might be skewed. And, it reinforces the point that the mean is a known value while the median is sensitive to other properties, one if which being the n.

This analysis is similar to Miller, J. (1988). A warning about median reaction time. Journal of Experimental Psychology: Human Perception and Performance, 14(3):539–543.

REVISION

Upon thinking about the skew issue I considered that the impact on the median might just be because in small samples you have a greater probability that the median is in the tail of the distribution, whereas the mean will almost always be weighted by values closer to the mode. Therefore, perhaps if one was just sampling with a probability of outliers then maybe the same results would occur.

So I thought about situations where outliers may occur and experimenters may attempt to eliminate them.

If outliers happened consistently, such as one in every single sampling of data, then medians are robust against the effect of this outlier and the conventional story about the use of medians holds.

But that's not usually how things go.

One might find an outlier in very few cells of an experiment and decide to use median instead of mean in this case. Again, the median is more robust but it's actual impact is relatively small because there are very few outliers. This would definitely be a more common case then the one above but the effect of using a median would probably be so small that it wouldn't matter much.

Perhaps more commonly outliers might be a random component of the data. For example, the true mean and standard deviation of the population may be about 0 but there's a percentage of the time we sample from an outlier population where the mean is 3. Consider the following simulation, where just such a population is sampled varying the sample size.

#generate n samples N times with an outp probability of an outlier.
rout <- function (n, N, outp) {
outPos <- sample(0:1,n*N, replace = TRUE, prob = c(1-outp,outp))
numOutliers <- sum(outPos)
y <- matrix( rnorm(N*n), ncol = N )
y[which(outPos==1)] <- rnorm(numOutliers, 4)
return(y)
}

outp <- 0.1
N <- 1e4
ns <- 3:30
yMed <- sapply(ns, function(x) mean(apply(rout(x,N,outp), 2, median)))
var(yMed)
yM <- sapply(ns, function(x) mean(colMeans(rout(x,N,outp))))
var(yM)
plot(ns,yMed, type = 'l', ylim = range(c(yMed,yM)), ylab = 'Y', xlab = 'n', col = 'red')
lines(ns,yM)


The median is in red and mean in black. This is a similar finding to that of a skewed distribution.

In a relatively practical example of the use of medians to avoid the effects of outliers one can come up with situations where the estimate is affected by n much more when the median is used than when the mean is used.

-
Nice example, but it really depends on the distribution. If you use a normal distribution or an uniform distribution the graph is very different, with the two lines being superimposed. It's the exponential distribution that produces the difference. –  nico Aug 13 '11 at 11:55
-1 This answer confuses "sensitivity" with "bias." –  whuber Aug 13 '11 at 14:30
Nico, yes, they're superimposed with symmetric distributions, that's the point. One often uses the median in situations of a skew to reduce the influence of that skew on the results. But if the n's are small, and unequal then it also adds a bias.... which is a worse situation. –  John Aug 13 '11 at 14:37
@John Could you explain exactly how your graph exhibits "sensitivity to outliers"? All I see is that it shows that the sample mean is an unbiased estimator of the true mean (1.0) whereas the sample median is asymptotically an unbiased estimator of the true median (about 0.875798), with a slight positive bias for small $ns$. –  whuber Aug 13 '11 at 14:53
Much better; I've removed the downvote. But I'm intrigued by the new explanation: could you point out some source--a text, paper, or Web site--that actually makes the claim "[the median] is also considered robust to bias in skewed distributions" and explains what that might mean? I haven't come across such a claim before and am not sure what it's really saying. –  whuber Aug 14 '11 at 23:04
• From the mean it's easy to calculate the sum over all items, e.g. if you know the average income of the population and the size of the population, you can immediately calculate the total income of the entire population.

• The mean is straightforward to calculate in O(n) time complexity. Calculating the median in linear time is possible but requires more thought. The obvious solution requiring sorting has worse (O(n log n)) time complexity.

And I speculate that there is another reason for the mean being more popular than the median:

• The mean is taught to more persons at school and it's probably taught before teaching the median
-
For your time complexity point, it depends on how the values are stored. If the values are already sorted, then it is certainly possible to calculate median in O(1) worst case time complexity. –  luiscubal Aug 13 '11 at 15:58
I agree - its applicability in calculations like sums is one of the main advantages of the mean. While I often prefer the median when the goal is to describe something, we often use the mean when it is an input to another calculation. –  Jonathan Nov 23 '11 at 20:13

If the concern is over the presence of outliers, there are some straight-forward ways to check your data.

Outliers, almost by definition, come into our data when something changes either in the process generating the data or in the process collecting the data. i.e. the data ceases to be homogeneous. If your data is not homogeneous then neither the mean nor the median make much sense, since you are trying to estimate the central tendency of two separate data sets that have been mixed together.

The best method to ensure homogeneity is to examine the data-generating and -collection processes to ensure that all of your data is coming from a single set of processes. Nothing beats a little brain-power, here.

As a secondary check, you can turn to one of several statistical tests: chi-squared, Dixon's Q-test, Grubb's test or the control chart / process behavior chart (typically X-bar R or XmR). My experience is that, when your data can be ordered as it was collected, the process behavior charts are better at detecting outliers than the outlier tests. This use for the charts may be somewhat controversial, but I believe it is entirely consistent with Shewhart's original intent and it is a use that is explicitly advocated by Donald Wheeler. Whether you use the outliers tests or the process behavior charts, remember that a detected "outlier" is merely signalling potential non-homogeneity that needs to be further examined. It rarely makes sense to throw out data points if you don't have some explanation for why they were outliers.

If you are using R, the outliers package provides the outliers tests, and for process behavior charts there is the qcc, IQCC and qAnalyst. I have a personal preference for the usage and output of the qcc package.

-