When does the amount of skew or prevalence of outliers make the median preferable to the mean?

Question

I know that "deviations in the data are devil", and when the distribution is highly skewed, it is better to consider median as average rather than mean, but how to decide these hard-limits.

For example:

• CASE 1:

  • Assume X = 10,20,30,40,50,60,70
  • In this case, I think that it is better to use mean and that it will give very accurate results.

• CASE 2:

  • Assume X = 10,20,30,40,50,60,70,7000
  • In this case, I think that it is better to use median instead of using the mean.

• CASE 3:

  • Assume X = 10,20,30,400,500,600,700
  • In this case, I think it is better to use IQR (Inter Quartile Range)

But I'm stuck with how to decide these hard-limits i.e. which to use in which condition, in general.

I've found a tool working on subjected principle, which takes context-less sample-distribution as input and determines whether mean is close/moderate or against the null-hypothesis.

Find References:-

• http://home.ubalt.edu/ntsbarsh/Business-stat/otherapplets/MeanTest.htm (For Mean Test)
• http://home.ubalt.edu/ntsbarsh/Business-stat/otherapplets/MediansTest.htm (For Median Test)

What I'm really looking is a good answer which states how to derive these conclusions.

Answer 1

Framing the question

• You are asking an applied and subjective question, and thus, any answer needs to be infused with applied and subjective considerations.

• From a purely statistical perspective, the mean and median both provide different information about the central tendency of a sample of data. Thus, neither is correct or incorrect by definition.

• From an applied perspective, we often want to say something meaningful about the central tendency of a sample, where central tendency maps onto some subjective notion of "typical".

General thoughts

• When summarising what is typical in a sample, observations that are many standard deviations away from the mean (perhaps 3 or 4 SD) will have a large influence on the mean, but not the median. Such observations may lead the mean to deviate from what we think of as the "typical" value of the sample. This helps to explain the popularity of the median when it comes to reporting house prices and income, where a single island in the pacific or billionaire could dramatically influence the mean, but not the median. Such distributions can often include extreme outliers, and the distribution is positively skewed. In contrast, the median is robust.

• The median can be problematic when the data takes on a limited number of values. For example, the median of a 5-point Likert item lacks the nuance possessed by the mean. For example, means of 2.8, 3.0, and 3.3 might all have a median of 3.

• In general, the mean has the benefit of using more of the information from the data.

• When skewed distributions exist, it is also possible to transform the distribution and report the mean of the transformed distribution.

• When a distribution includes outliers, it is possible to use a trimmed mean, or remove the outliers, or adjust the value of the outlier to a less extreme value (e.g., 2 SD from the mean).

Answer 2

You can read about measures of central tendency here: http://en.wikipedia.org/wiki/Central_tendency .

Generally, you analyse a sample in order to tell something about a (much larger) population. Often you know more about the population than merely the data in your sample, usually something that motivated you to take a sample in the first place. If you know that the population has normal distribution then the sample mean will be the best estimator of the expected value even if the sample does not look normal (using small sample sizes like the above you can't really characterise the distribution anyway). You can reliably estimate the mean if you have a lot of data even if the distribution is not normal (see T-test for non normal when N>50?).

In cases of distributions that can not be described parametrically median and the IQR may tell much more. IQR is a dispersion measure as opposed to the mean and median that are location measures. You can read about dispersion parameters here: http://en.wikipedia.org/wiki/Statistical_dispersion .

A further aspect to consider is that some of your data may be outliers (see Rigorous definition of an outlier?).

Answer 3

There are no hard and fast rules. They convey different information and have different properties. You select the statistic that best conveys what you want to convey. Or better yet, select statistics that best describe the data. Keep this same thing in mind when you're selecting a measure of central tendency to analyze.

(snipped a bunch of stuff repeating Mike Lawrence's answer)

Note that Mike Lawrence is referring to something that's surprising for a lot of people. In the behavioural sciences there's a lot of folk wisdom that you use medians with small sample sizes. But in actual fact that's exactly the wrong thing to do because the median quickly becomes more biased than the mean with small samples.

Answer 4

Be careful with medians: they are biased estimators and the degree of bias can change depending on the skew of the distribution and the sample size (see Miller, 1988). This means that if you are comparing two conditions that have either different skew or different sample sizes, you may find a difference that is in fact attributable to bias rather than a real difference, or you may fail to find a difference when there is a real one when the direction of the difference in bias is opposite to the direction of the real difference between the conditions.

Answer 5

"deviations in the data are the devil" is just not true I think - well I don't agree with it at least. I'd say its more like "chilli" than the "devil" - as much as you can reasonably handle is good, but it can get nasty if there is too much.

The most general procedure I know of to "choose a statistic" to report your data is a combination of two things

1. Bayesian inference (describing what is known)
2. Decision theory (taking actions under uncertainty)

However, both of these methods are only partially "algorithmic" so to speak. You have to supply the inputs though. Perhaps the most important part of this stage is that you have to ask a question that your procedure is going to answer. Naturally, different questions get different answers. As the saying goes "I have just derived a very elegant and beautiful answer. All I have to do now is figure out the question." This is a common problem that I have seen with many statistical procedures, is that there is not always a clear statement of the class of problems that it is the best procedure to use.

Bayesian inference requires you to specify your prior information in a mathematical framework. This involves

1. Specifying the hypothesis space - what possibilities am I going to consider?
2. Assigning probabilities to each part of the space
3. Using the rules of probability theory to manipulate the assigned probabilities

this is basically an open-ended problem (you can always analyse a given English statement more deeply, to extract more or different information from it). Decision theory also requires you to specify a loss function - and there are basically no rules or principles by which to do this, at least as far as I know (computational simplicity is a key driver).

One useful question to ask yourself though is "what information about the sample do I convey by presenting this statistic?" or "how much of the complete data set can I recover from using just this set of statistics?"

One way you could use Bayesian statistics to help you here is to propose a hypothesis:

$$\begin{array}{l l} H_{mean}:\text{The mean is the best statistic} \\H_{med}:\text{The median is the best statistic} \\H_{IQR}:\text{The IQR is the best statistic} \end{array} $$

Now these are not "mathematically well posed" hypothesis, but if we use them anyway, and see what parts of the maths are required to make it well posed. The first part is the prior probabilities, without any data, how likely is each hypothesis? The usual answer is equal probabilities (but not always - may have some theoretical reason to support one hypothesis being more likely - the CLT is perhaps one for $H_{mean}$ being higher than the others).

So we use Bayes theorem to update each probability ($I$=prior information, $D$= data set):

$$P(H_{i}|D,I)=P(H_{i}|I)\frac{P(D|H_{i},I)}{P(D|I)}\implies \frac{P(H_{i}|D,I)}{P(H_{j}|D,I)}=\frac{P(H_{i}|I)}{P(H_{j}|I)}\frac{P(D|H_{i},I)}{P(D|H_{j},I)}$$

So if the prior probabilities are equal, then the relative probabilities are given by the likelihood ratio. So you also need to specify a probability distribution for what type of data sets you would be likely to see if the mean was the best statistic, etc. Note that each hypothesis doesn't actually state what the specific value of the mean, median, or IQR actually is. Therefore, the probability cannot depend on the exact value of the mean. Hence in the likelihoods these must have been "integrated out" using the sum and product rules

$$P(D|H_{i},I)=\int P(\theta_{i}|H_{i},I)P(D|\theta_{i},H_{i},I)d\theta_{i}$$

So you have the prior $P(\theta_{i}|H_{i},I)$ which can be interpreted for i=mean as "given that the mean is the best statistic, and prior to seeing the data, what values of the mean are we likely to see?" and the likelihood $P(D|\theta_{i},H_{i},I)$ can be similarly interpreted as "given the mean is best, and equal to $\theta_{mean}$ how likely is the data that was observed?". This may help you come up with some kinds of features that your distribution should have.

This describes the inference - now it is time to apply decision theory. This is particularly simple because your decision doesn't influence the state of nature - the statistic won't change if you do or don't use it. So we can describe the decisions ($A$ for "action" because $D$ is already taken):

$$\begin{array}{l l} A_{mean}:\text{The mean is the reported statistic} \\A_{med}:\text{The median is the reported statistic} \\A_{IQR}:\text{The IQR is the reported statistic} \end{array} $$

And now you need to specify a loss matrix $L_{ij}$ which relates the action/decision $A_{i}$ to the state of nature $H_{j}$ - what is the loss if I report the mean, but the median is actually the best statistic? In most cases the diagonal elements will be zero - taking the correct action means no loss. You may also have that all non-diagonal elements are equal - how you are wrong doesn't matter, only whether or not you are wrong.

You then proceed by calculating the average loss for each action, weighted by their probabilities:

$$L_{i}=\sum_{j}L_{ij}P(H_{j}|D,I)$$

And you then choose the action with the smallest average loss.

Answer 6

In your examples you seem to make the underlying assumption that you are "blind" to how the data you are going to analyze were generated. In other words, whether mean or median is the right tool for say something robust about central tendency of a sample of data largely depends on your (prior) knowledge of the data generating process. That's probably the main reason why most of us dealing with statistics are steadily focused on the same research fields (or their nearest outskirsts). As a sidelight, in real practice the 7000 value of your in your CASE 2 example would sound more like a mistaken data entry than a genuine outlier.