Skip to main content
25 votes
Accepted

Non-parametric test if two samples are drawn from the same distribution

The Kolmogorov-Smirnov test is the most common way to do this, but there are also some other options. The tests are based on the empirical cumulative distribution functions. The basic procedure is: ...
Will's user avatar
  • 1,078
25 votes
Accepted

Why the p-value of t.test() is not statistically significant when mean values look really different

I agree with @pikachu that the standard deviations are too large compared with the difference between means for a t test to find a significant difference. Thank you for posting your data. It is always ...
BruceET's user avatar
  • 57k
20 votes
Accepted

Significant Mann Whitney, and significant t-test, but in the other direction

(via Mann Withney, or Kruskal Wallis), While the test statistic in Mann-Whitney does correspond to a measure of $P(X>Y)$, once you go beyond two samples Kruskal-Wallis doesn't quite do that in ...
Glen_b's user avatar
  • 286k
18 votes
Accepted

Can I use a Mann-Whitney U Test with a very small sample?

This has been discussed at length on this site. Briefly, the test is valid. But no test is especially helpful because of our inability to interpret large p-values, which do not indicate "no ...
Frank Harrell's user avatar
15 votes
Accepted

Why does R say 'cannot compute exact p-values with ties' when I can do it with pen and paper?

The wilcox.test from the standard stats library is limited to cases without ties because it uses an algorithm from the function <...
Sextus Empiricus's user avatar
14 votes
Accepted

When should I use `scipy.stats.wilcoxon` instead of `scipy.stats.ranksums`?

Frank Wilcoxon's 1945 paper [1] described two tests -- for "Unpaired Experiments" and "Paired Comparisons" which have come to be called the (Wilcoxon) rank sum test and the (Wilcoxon) signed rank test ...
Glen_b's user avatar
  • 286k
14 votes

Kolmogorov Smirnov Z vs Mann Whitney U small sample size n= 15?

If the original statement doesn't limit the conditions under which it applies pretty substantially, Field is just wrong on this. Responding to the quoted section: In effect, this means it does much ...
Glen_b's user avatar
  • 286k
14 votes

Detect if there is actually two populations in a sample

There is no way to do this by non-parametric paradigm, just think of it: the sampled distribution is a completely legit one, there is nothing preventing a single-population distribution from having ...
carlo's user avatar
  • 4,545
14 votes

Can I use a Mann-Whitney U Test with a very small sample?

Frank's advice is useful; I don't wish my answer to suggest any disagreement with that answer. The Wilcoxon-Mann-Whitney test "works as it should" in small samples. There's a few things to ...
Glen_b's user avatar
  • 286k
13 votes

What is the interpretation of the p-value of 2.2e-16?

2.2e-16 is the scientific notation of 0.00000000000000022, meaning it is very close to zero. Your statistical software probably uses this notation automatically for very small numbers. You may be able ...
morphist's user avatar
  • 444
13 votes
Accepted

Using wilcox.test() and t.test() in R yielding different p-values

Steady on there! You have two very small samples there. Statistics is not taught at Hogwarts! No white magic for very small samples. Not rejecting the null on Shapiro-Wilk doesn't allow the ...
Nick Cox's user avatar
  • 58.6k
11 votes

Mann-Whitney U Test Definition of U

Anyone is allowed to give any possible name for the test statistic they design. Mann and Whitney (1947) decided to name the one they proposed as "$U$". In their paper they do not give any reasons why ...
Tim's user avatar
  • 140k
11 votes
Accepted

Always use Welch-t test (unequal variances t-test) instead of Student-t or Mann-Whitney test?

There have been a number of papers which examine this issue. Most of them come to the conclusion that Welch's version of the t-test can be safely used in most circumstances. The only situation in ...
klumbard's user avatar
  • 1,401
11 votes
Accepted

Why was I told to use a sample size of 30 when using the Mann–Whitney U test?

The Mann Whitney test does not require any specific N. However, what your instructor is probably talking about is power; that is, with a small N, differences are not going to be statistically ...
Peter Flom's user avatar
  • 125k
11 votes

Why the p-value of t.test() is not statistically significant when mean values look really different

When you consider the difference between means you have to use a different unit than the simple absolute difference. Take into account that you are measuring the difference in means produced by two ...
rapaio's user avatar
  • 7,094
11 votes
Accepted

What can cause exploding statistic values and p-values near zero with the Wilcoxon-Mann-Whitney test?

Nothing out of the ordinary is going on from the sound of it. In almost all cases, I get huge values for the statistic Have you looked at the range of possible values for the statistic? For the ...
Glen_b's user avatar
  • 286k
11 votes

Mann Whitney test problem

Welcome to CV. First, when you have N = 16, any test of normality is likely to be fairly useless. How did you determine normality? Second, while we can't tell exactly what is going on without the data,...
Peter Flom's user avatar
  • 125k
11 votes

Mann Whitney test problem

A couple of points to supplement what Frank and Peter wrote. The normality assumption is that your data are sampled from distributions (or populations) that are (approximately) normal. It makes no ...
Harvey Motulsky's user avatar
10 votes
Accepted

Interpretation of p-value in Mann-Whitney rank test

The p-value represents the probability of getting a test-statistic at least as extreme$^\dagger$ as the one you had in your sample, if the null hypothesis were true. A high p-value indicates you saw ...
Glen_b's user avatar
  • 286k
10 votes
Accepted

Mann-Whitney-Wilcoxon Test on highly unbalanced groups.

In no way will the difference in sample sizes adversely affect the Mann-Whitney-Wilcoxon test. It's explicitly suitable for groups of different sizes, and how different doesn't impact the essential ...
Glen_b's user avatar
  • 286k
10 votes

Is the Wilcoxon rank-sum test a nonparametric alternative to the two sample t-test? Null hypotheses are different

Is the Wilcoxon rank-sum test a nonparametric alternative to the two sample t-test? Yes and no. (Go not to the elves for counsel...) Speaking broadly, any given test statistic has some power curve ...
Glen_b's user avatar
  • 286k
10 votes
Accepted

Detect if there is actually two populations in a sample

Let's start with terminology. Population in statistics is the "set of entities under study". When designing the study, we define the population of interest and then draw samples from this population. ...
Tim's user avatar
  • 140k
10 votes
Accepted

What does comparing mean rank mean?

When comparing two independent samples, you want to rank all the data together. Revising your example: Sample A ...
Alexis's user avatar
  • 30.3k
10 votes
Accepted

Should I make equally sized samples for the Mann-Whitney U test if originally I have unequal sample sizes

Use your original data. Claims about losing power when groups have unequal sizes make more sense in the context of designing the experiment. If you have enough resources to make $100$ observations, ...
Dave's user avatar
  • 65.1k
10 votes

Mann Whitney test problem

As Peter implied, trusting the data to inform you about normality is trusting them too much. And it implies that you think that the Wilcoxon or Kruskal-Wallis tests don’t work well with normal data ...
Frank Harrell's user avatar
9 votes

Where does the number 12 come from and why is it the same 12 in Wilcoxon and Kruskal Wallis?

In both cases 12 appears when approximating the distribution of the test statistic with a normal and chi-square respectively because the statistic must first be written in a standardized form (if you ...
Glen_b's user avatar
  • 286k
9 votes

Can the Wilcoxon rank sum test give a different result to the Kolmogorov-Smirnov test result?

Statistical tests measure differences between samples in different ways and are sensitive to different aspects of the distributional differences. It's certainly possible for one test to reject and ...
Glen_b's user avatar
  • 286k
9 votes
Accepted

Mann Whitney Test: Clearing Up Confusion

It's quite reasonable to report the medians when doing a Mann-Whitney test. The median is perfectly good location summary. The problem is describing the test as a test for difference in median. I ...
Thomas Lumley's user avatar
9 votes

Detect if there is actually two populations in a sample

In statistical terms, you are wondering whether your data comes from a mixture of two (or more) populations, as against coming from a single population. Looking at the mixture or more specifically the ...
Stephan Kolassa's user avatar
9 votes
Accepted

What difference between Mann-whintey U-test and Kolmogorov-Smirnov test on truncated log normal distributions?

If I understand the documentation of stats.ks_2samp correctly, alternative="less" means that the cdf of data A is smaller that the one of data B. This means that the distribution with cdf A ...
Christian Hennig's user avatar

Only top scored, non community-wiki answers of a minimum length are eligible