# Non-parametric tests comparing dependent samples

Suppose I have two samples S1 and S2, but they are dependent samples. My aim is to compare if the mean of S1 is significantly smaller than the mean of S2. The samples are not normally distributed, so because of the distributional assumption, a non-parametric test (Wilcoxon sum-rank test, say) is preferred. However, these kind of non-parametric tests still require the samples to be independent.

My question is, is there a test that deals with dependent samples? How about sampled permutation test? For example, I have 5 data points in S1 and 10 data points in S2, and I can do the following (set counter = 0):

1. Reshuffle the 15 points into 2 groups, each having 5 and 10 points
2. Calculate the means of group 1 and group 2, and denote the difference as m1
3. Compare with the observed mean difference m0: if m0 < m1, counter = counter + 1

Repeat the steps above for N=1000 times, and the p-value is counter/N.

Will this sampled permutation test handle dependent data well, compared with a single non-parametric test? Thanks!

Your permutation test is appropriate for a one-tailed test of a mean difference between independent samples. It will be underpowered since it ignores the dependence structure. For example, consider an experiment in which 10 rats were run through a maze twice, with and without a drug. So the $i^{th}$ rat has two measurements, completion time with the drug, $t_D^{(i)}$, and time without, $t_C^{(i)}$. Each iteration of a permutation test in this case would randomly swap scores within each rat, not across rats as in your example. In other words, $t_C^{(i)}$ is swapped with $t_D^{(i)}$ with probability 0.5 for each $i$, but $t_C^{(i)}$ is never swapped with $t_D^{(j)}$. Simply shuffling all scores into two groups of ten would result in a test that suffers from reduced power relative to the more appropriate test.