Is median fairer than mean?

Question

I recently read the advice that you should generally use median not mean to eliminate outliers. Example: The following article http://www.amazon.com/Forensic-Science-Introduction-Scientific-Investigative/product-reviews/1420064932/

has 16 reviews at the moment:

review= c(5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 4, 4, 3, 2, 1, 1)
summary(review)  ## "ordinary" summary

Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
1.000   3.750   5.000   4.062   5.000   5.000 

Because they use Mean the article gets 4 stars but if they would use Median it would get 5 stars.

Isn't the median a 'fairer' judge?

---

An experiment shows that median's error is always bigger than mean. Is median worse?

library(foreach)

#the overall population of bookjudgments
n<-5
p<-0.5
expected.value<-n*p
peoplesbelieve <-rbinom(10^6,n, p)

#16 ratings made for 100 books
ratings <- foreach(i=1:100, .combine=rbind) %do% sample(peoplesbelieve,16)
stat <- foreach(i=1:100, .combine=rbind) %do% c(mean=mean(ratings[i,]), median=median(ratings[i,]))

#which mean square error is bigger? Mean's or Median's?
meansqrterror.mean<-mean((stat[,"mean"]-expected.value)^2)
meansqrterror.median<-mean((stat[,"median"]-expected.value)^2)

res<-paste("mean MSE",meansqrterror.mean)
res<-paste(res, "| median MSE", meansqrterror.median)
print(res)

Answer 1

The problem is that you haven't really defined what it means to have a good or fair rating. You suggest in a comment on @Kevin's answer that you don't like it if one bad review takes down an item. But comparing two items where one has a "perfect record" and the other has one bad review, maybe that difference should be reflected.

There's a whole (high-dimensional) continuum between median and mean. You can order the votes by value, then take a weighted average with the weights depending on the position in that order. The mean corresponds to all weights being equal, the median corresponds to only one or two entries in the middle getting nonzero weight, a trimmed average corresponds to giving all except the first and last couple the same weight, but you could also decide to weight the $k$th out of $n$ samples with weight $\frac{1}{1 + (2 k - 1 - n)^2}$ or $\exp(-\frac{(2k - 1 - n)^2}{n^2})$, to throw something random in there. Maybe such a weighted average where the outliers get less weight, but still a nonzero amount, could combine good properties of median and mean?

Answer 2

The answer you get depends on the question you ask.

Mean and median answer different questions. So they give different answers. It's not that one is "fairer" than another. Medians are often used with highly skewed data (such as income). But, even there, sometimes the mean is best. And sometimes you don't want ANY measure of central tendency.

In addition, whenever you give a measure of central tendency, you should give some measure of spread. The most common pairings are mean-standard deviation and median-interquartile range. In these data, giving just a median of 5 is, I think, misleading, or, at least, uninformative. The median would also be 5 if every single vote was a 5.

Answer 3

If the only choices are integers in the range 1 to 5, can any really be considered an outlier?

I'm sure that with small sample sizes, popular outlier tests will fail, but that just points out the problems inherent in small samples. Indeed, given a sample of 5, 5, 5, 5, 5, 1, Grubbs' test reports 1 as an outlier at $\alpha = 0.05$. The same test for the data you give above does not identify the 1's as outliers.

Grubbs test for one outlier

data:  review  G = 2.0667, U = 0.6963,
p-value = 0.2153 alternative
hypothesis: lowest value 1 is an outlier

Answer 4

Just a quick thought:

If you assume that each rating is drawn from a latent continuous variable, then you could define the median of this underlying continuous variable of interest as your value of interest, rather than the mean of this underlying distribution. Where the distribution is symmetric, then the mean and the median would ultimately be estimating the same quantities. Where the distribution is skewed, the median would differ from the mean. In this case, to my mind, the median would correspond more with what we think of as the typical value. This goes some way to understanding why median income and median house prices are typically reported rather than the mean.

However, when you have a small number of discrete values, the median performs poorly.

Perhaps, you could use some density estimation procedure and then take the median of that, or use some interpolated median.