# Is median fairer than mean?

I recently read the advice that you should generally use median not mean to eliminate outliers. Example: The following article http://www.amazon.com/Forensic-Science-Introduction-Scientific-Investigative/product-reviews/1420064932/

has 16 reviews at the moment:

review= c(5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 4, 4, 3, 2, 1, 1)
summary(review)  ## "ordinary" summary

Min. 1st Qu.  Median    Mean 3rd Qu.    Max.
1.000   3.750   5.000   4.062   5.000   5.000


Because they use Mean the article gets 4 stars but if they would use Median it would get 5 stars.

Isn't the median a 'fairer' judge?

An experiment shows that median's error is always bigger than mean. Is median worse?

 library(foreach) #the overall population of bookjudgments n<-5 p<-0.5 expected.value<-n*p peoplesbelieve <-rbinom(10^6,n, p) #16 ratings made for 100 books ratings <- foreach(i=1:100, .combine=rbind) %do% sample(peoplesbelieve,16) stat <- foreach(i=1:100, .combine=rbind) %do% c(mean=mean(ratings[i,]), median=median(ratings[i,])) #which mean square error is bigger? Mean's or Median's? meansqrterror.mean<-mean((stat[,"mean"]-expected.value)^2) meansqrterror.median<-mean((stat[,"median"]-expected.value)^2) res<-paste("mean MSE",meansqrterror.mean) res<-paste(res, "| median MSE", meansqrterror.median) print(res) 

-
Why would having a 5 star rating be fairer? There are 6/16 reviews that gave a lower rating... – nico Feb 5 '11 at 17:00
Ok, than you think Mean is the right average? the majority said its 5. 60% more the rest 6/16 said so. – Roland Kofler Feb 5 '11 at 17:08
If I had to give a discrete evaluation, seeing those 16 reviews I would give 4, not 5, as -to me- 5 would mean that all (or the great majority) of the votes are 5. 6/16 is ~40%, which is not exactly negligible. – nico Feb 5 '11 at 18:22
So, in essence, I think neither mean or median are good. Showing (as Amazon does) a bargraph with the different votes is the best option. Also, it is interesting to point out that online 1-5 ratings are not always so fair... youtube-global.blogspot.com/2009/09/… – nico Feb 5 '11 at 18:27
@nico: scoring is full of traps, what you point out is one of the arguments of my article here: objektorient.blogspot.com/2010/09/… – Roland Kofler Feb 5 '11 at 19:56
show 1 more comment

The problem is that you haven't really defined what it means to have a good or fair rating. You suggest in a comment on @Kevin's answer that you don't like it if one bad review takes down an item. But comparing two items where one has a "perfect record" and the other has one bad review, maybe that difference should be reflected.

There's a whole (high-dimensional) continuum between median and mean. You can order the votes by value, then take a weighted average with the weights depending on the position in that order. The mean corresponds to all weights being equal, the median corresponds to only one or two entries in the middle getting nonzero weight, a trimmed average corresponds to giving all except the first and last couple the same weight, but you could also decide to weight the $k$th out of $n$ samples with weight $\frac{1}{1 + (2 k - 1 - n)^2}$ or $\exp(-\frac{(2k - 1 - n)^2}{n^2})$, to throw something random in there. Maybe such a weighted average where the outliers get less weight, but still a nonzero amount, could combine good properties of median and mean?

-

Mean and median answer different questions. So they give different answers. It's not that one is "fairer" than another. Medians are often used with highly skewed data (such as income). But, even there, sometimes the mean is best. And sometimes you don't want ANY measure of central tendency.

In addition, whenever you give a measure of central tendency, you should give some measure of spread. The most common pairings are mean-standard deviation and median-interquartile range. In these data, giving just a median of 5 is, I think, misleading, or, at least, uninformative. The median would also be 5 if every single vote was a 5.

-
Your point about a measure of spread is a key here. That's one of the issues that keeps coming up in this discussion, under other names, and it also ties in with Erik P's discussion of weighting schemes. – Wayne Feb 6 '11 at 22:47

If the only choices are integers in the range 1 to 5, can any really be considered an outlier?

I'm sure that with small sample sizes, popular outlier tests will fail, but that just points out the problems inherent in small samples. Indeed, given a sample of 5, 5, 5, 5, 5, 1, Grubbs' test reports 1 as an outlier at $\alpha = 0.05$. The same test for the data you give above does not identify the 1's as outliers.

Grubbs test for one outlier

data:  review  G = 2.0667, U = 0.6963,
p-value = 0.2153 alternative
hypothesis: lowest value 1 is an outlier

-
because just one person in your example can change the result dramatically. if the person believed the book has a different topic his fault will change the rating – Roland Kofler Feb 5 '11 at 17:11
Is someone's opinion a fault? I'd argue that the failure is trying to draw meaningful conclusions based on a single statistic from only a few data points. As noted by @nico above, Amazon does show a bar graph of all the ratings. – kmm Feb 7 '11 at 16:14

Just a quick thought:

If you assume that each rating is drawn from a latent continuous variable, then you could define the median of this underlying continuous variable of interest as your value of interest, rather than the mean of this underlying distribution. Where the distribution is symmetric, then the mean and the median would ultimately be estimating the same quantities. Where the distribution is skewed, the median would differ from the mean. In this case, to my mind, the median would correspond more with what we think of as the typical value. This goes some way to understanding why median income and median house prices are typically reported rather than the mean.

However, when you have a small number of discrete values, the median performs poorly.

Perhaps, you could use some density estimation procedure and then take the median of that, or use some interpolated median.

-