3
$\begingroup$

I have two sets of results from an experiment that produces distributions with extremely heavily stacked sides and mostly uniform elsewhere. The aim of my analysis is to answer a the question roughly put like this : "With what degree of certainty can I say that the difference between Results A and Results B are not due to noise in the measurement/system"

The Wilcoxon Rank-Sum produces a highly significant p-value although one would expect that this is coming solely from the fact data A is more stacked at zero than data B. It may be the case that the distributions are significantly different but I was wondering if there might be a more appropriate test than the Wilcoxon Rank-Sum test given the shape of the data?

x
x
x                          x
x                          x
x                          x                                       
x                   x      x                   x
x                   x      x                   x
x                   x      x                   x
x                   x      x                   x
x     x   x x       x      x   x   x   x   x   x
x x x x x x x x x   x      x x x x x x x x x x x
x x x x x x x x x x x      x x x x x x x x x x x
0-------------------1      0-------------------1 
     A                               B

In the actual data the sides are even more heavily stacked than in the diagram.

$\endgroup$

1 Answer 1

2
$\begingroup$

If you want to test if two samples came from the exact same distributoin, the you can use the nonparametric Two-sample KS Test. It assumes the underlying distribution is continuous.

Another option for esting quality is the permutation test, where you pool both samples into one large sample and cycle through all possible ways of dividing it, each time calculating your test statistic. P-value is just the number of permutations that generate test statistics at least as extreme as the one you got using the actual samples for each.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.