Apologies if this is a very basic question.

If we have data that are not normally distributed (e.g. skewed, Shapiro-Wilk test is significant) and we resort to rank-based methods (e.g. Wilcoxon Signed Rank test), then do we need to be concerned with outliers?

Imagine, for example, we plot the data using a boxplot and a minority of data points are marked as outliers. Should we transform those points? Or remove them? It seems to me that many textbooks talk about dealing with outliers, but only because they exert a major influence on the parameters such as mean and standard deviation. However, when we use a rank-based test they will already be 'transformed' to be the next value in the rank, and would therefore not exert a major influence on the test. I have not seen this stated explicitly in a statistics book so far, so I thought I would ask the question here.

Do we need to worry about outliers when using rank-based tests?

  • 3
    $\begingroup$ Statistically rank-based tests are robust against outlier. But an outlier is an outlier, on the operational level the analyst should still examine that case. So, I'd say we still need to "partially" worry about outliers. $\endgroup$ Commented Sep 25, 2013 at 12:53
  • $\begingroup$ It's fine even if the question might be very basic. As long as the question is unasked on this site, even basic questions are good questions $\endgroup$
    – Hotaka
    Commented Sep 25, 2013 at 13:28

2 Answers 2


No. When the data are ranked, an outlier will simply be recognized as a case that is ranked one above (or below) the next less extreme case. Regardless of whether there is .01 or 5 standard deviations between the most and second most extreme value, that degree of difference is thrown away when data are ranked.

In fact, one of the many reasons why someone might use a rank-based (or nonparametric) test is because of outliers.

  • 3
    $\begingroup$ And the excellent efficiency of nonparametric and semiparametric methods is a reason not to pre-test for normality (besides the inadequate power of the normality test). $\endgroup$ Commented Sep 25, 2013 at 11:45
  • $\begingroup$ I'm at a loss as to whether to accept this great concise answer or the thought-provoking detailed one from @NickCox. I will wait a couple of days to see which ends up with the most votes! $\endgroup$
    – trev
    Commented Sep 26, 2013 at 8:44

@Hotaka's answer is quite correct. Ranking makes transformation unnecessary; it is itself a transformation that ignores exact values except in so far as they lead to differences in rank. In fact, a little thought, or some example calculations, will show that the results after ranking logarithms or square roots or any other monotonic transformation are exactly the same as those after ranking the original data.

But more can be said. The either-or thinking

  • Either my data are normally distributed, and I can use standard or classical procedures.

  • Or I need to resort to rank-based tests.

is a little stark, and (it may be suggested) over-simplified. Although it's hard to suggest exactly what you should be doing without some sight of your data and your precise goals, there are other perspectives:

  1. Many users of statistics look at marginal (univariate) distributions and assess whether they are close to normality, but that may not even be relevant. For example, marginal normality is not required for regression-type procedures. For many procedures, it's how the means behave, not how the data behave, that is more important and closer to the main assumptions.

  2. Even (say) a significant result at conventional levels for a Shapiro-Wilk test is equivocal in terms of guiding later analysis. It just says "your distribution is detectably different from a normal distribution". That itself does not imply that the degree of non-normality you have makes whatever you have in mind invalid or absurd. It may just mean: go carefully, as underlying assumptions are not exactly satisfied. (In practice, they never are exactly satisfied, any way.) The habit to cultivate is that of thinking that all P-values are approximations. (Even when assumptions about distributions are not being made, assumptions about sampling or independence or error-free measurement are usually implicit.)

  3. Although many texts and courses imply otherwise, non-parametric statistics is something of a glorious dead end: there are a bundle of sometimes useful tests, but in practice you give up on most of the useful modelling that is central to modern statistics.

  4. Outliers are mentioned here, and they always deserve close attention. They should never be omitted just because they are inconvenient or appear to be the reason why assumptions are not satisfied. Sometimes analysis on a transformed scale is the best way forward. Sometimes a few mild outliers are not as problematic as less experienced users of statistics fear. With small samples, data will often look ragged or lumpy, even if the generating process is quite well-behaved; with large samples, a single outlier need not dominate the rest of the data.

  5. There is always the option of doing both kinds of tests, e.g. Student's t and Mann-Whitney-Wilcoxon. They don't ask exactly the same question, but it is often easy to see if they point in the same direction. That is, if a t test and the other test both give clear signals that two groups are different, you have some reassurance that your conclusion is well supported (and some defence against the sceptic who distrusts one or other procedure given a whiff of non-normality). If the two tests give very different answers, this in itself is useful evidence that you need to think very carefully about how best to analyse data. (Perhaps that massive outlier really does determine which way the answer comes out.)

With experience, users of statistics are often more informal than texts or courses imply they should be. If you talked through an analysis with them, you would often find that they make quick judgements such as "Sure, the box plots show some mild outliers, but with data like this analysis of variance should work fine" or "With skew that marked, a logarithmic scale is the only sensible choice". I don't think you will often find them choosing techniques based on whether a Shapiro-Wilk test is or is not significant at $P < 0.05$. Saying something like that may not help less experienced users much, but it seems truer than the idea that statistics offers exact recipes that must always be followed.

  • $\begingroup$ Thank you for your detailed answer. Regarding choice of method, I can believe most users of statistics are fairly exploratory during the first look at their data. But when they write an article, they need to justify which method they chose. I guess this partly depends on the field and whether we are more interested in modelling lots of data or testing a hypothesis. For the latter, a Shapiro-Wilk, how ever under-powered, must look better than reporting skewness without a test, and then going on to conduct a non-parametric test. $\endgroup$
    – trev
    Commented Sep 26, 2013 at 0:35
  • 1
    $\begingroup$ I quite like the idea of (5), doing both parametric and non-parametric tests. But I've rarely seen an article (at least in psychology) that says "here are the results of various alternative statistical tests." They just choose one method and report that, which can be problematic, because they can just choose the method that gives them a significant result, as highlighted in the psych science article here: bit.ly/15uTFlT $\endgroup$
    – trev
    Commented Sep 26, 2013 at 0:42
  • $\begingroup$ Of course, the alternative of reporting multiple methods will almost certainly lead to some ambiguity, i.e. some methods being significant and others not. How many would you need to conclude you've got an effect? 4 out of 5 significant? What about 3 out of 5? $\endgroup$
    – trev
    Commented Sep 26, 2013 at 0:51
  • 1
    $\begingroup$ Your thoughtful comments deserve very detailed discussion. My experience confirms that people in many fields are very concerned to show that there's a single correct analysis of a given data set, which is what they did. $\endgroup$
    – Nick Cox
    Commented Sep 26, 2013 at 5:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.