The first thing that comes to mind when comparing some property of two samples is probably the independent samples t-test. Sometimes, people point out that this has the inherent assumption of the t-distribution. Does any one know of a non-parametric alternative to the independent samples t-test?

  • $\begingroup$ It has been claimed that the t-test itself is non-parametric because it does not require normality of the data, but only relies on approximate normality of the sampling distributions of the two means. (The t-test makes no assumption about a t-distribution in the underlying data, by the way.) When you refer to an "alternative," do you mean a test for comparing means of two populations or would any test for comparing their locations suffice? There are plenty of the latter discussed in the nonparametric tag: have you followed those links in search of answers yet? $\endgroup$
    – whuber
    Feb 21, 2014 at 22:05

1 Answer 1


R. A. Fisher justified using the t-test because it was a reasonable approximation to the permutation test under certain conditions and the t-test was simpler to compute. With modern computers the computation of the permutation test (or the estimation by sampling from the permutation distribution) is no longer something to be avoided. The permutation test does not depend on the assumption of normality and is not restricted to comparing means (it can be used to compare means, but it can be used with medians, ranges, or other functions of the data that may be of interest).

So one non-parametric alternative is the permutation test (some of the other common non-parametric methods are actually special cases of a permutation test).


For small samples you can calculate every possible computation, but it is more common to use resampling techniques to estimate the permutation distribution. The basic steps:

  1. Choose the statistic of interest (difference of means, difference of medians, etc.) and compute it for the original data.
  2. Merge all the data together then randomly divide it into 2 groups of the same sizes as the original. Compute the desired test statistic on this random grouping.
  3. Repeat step 2 a bunch of times (until the total number of stats is around 1,000 or 10,000).
  4. Compare the original statistic to the distribution of the permuted statistics (the p-value is the proportion of test statistics that are as extreme or more extreme than the original).
  • $\begingroup$ Googling permutation tests leads me to re-sampling methods. I have used these for things like estimating the sampling variation, but not for hypothesis testing. Can you give me a hint or point me to some references regarding how exactly a permutation test will work? $\endgroup$
    – ryu576
    Feb 21, 2014 at 22:53
  • $\begingroup$ @ryu576, I added some detail in the question. $\endgroup$
    – Greg Snow
    Feb 21, 2014 at 23:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.