Paired or unpaired non-parametric 2 sample test? I've got a relative simple question. I'm doing an imaging study with a phantom. I've run the same phantom through the MRI scanner 5 times, each time running the same scan twice and only varying one parameter. I am trying to assess if varying this parameter (independent variable) changes my response variable (dependent variable). Do I assume this is a paired measurement because both tests were conducted sequentially in the same session? I get a p value of 0.0625 running the Wilcoxon matched-pairs signed rank test (a function of a low sample number I assume), vs 0.008 when the Mann-Whitney U test is run. Thanks for your help!
 A: A paired approach seems appropriate because of the
way data were collected. A paired
Wilcoxon test is essentially the same a one-sample Wilcoxon Signed Rank
test on the differences. 
Unfortunately, assuming no ties among
the five differences, the smallest possible P-value $1/16 = 0.0625$
for a 2-sided test occurs when all differences have the same sign. (By chance
alone there is 1 chance in 32 that all five differences would be
positive and the same that tall five would be negative: $1/32+1/32 = 1/16.)$
Here are computations in R statistical software, based on some
hypothetical data:
x1 = c(1,4,5,7,8)
x2 = c(12,18,17,20,23)
d = x2-x1; d
[1] 11 14 12 13 15

Identical P-values of $1/16$ for both versions of the Wilcoxon signed-rank test, using paired data:
wilcox.test(x1, x2, pair=T)

        Wilcoxon signed rank test

data:  x1 and x2
V = 0, p-value = 0.0625
alternative hypothesis: 
   true location shift is not equal to 0

wilcox.test(d)

        Wilcoxon signed rank test

data:  d
V = 15, p-value = 0.0625
alternative hypothesis: 
    true location is not equal to 0

Doing a 2-sided Mann-Whitney-Wilcoxon two-sample rank sum test,
there are ten observations altogether, then if (as for my illustrative hypothetical data) all observations in one group are larger than any value in the other a smaller P-value is possible. Specifically, the P-value for
a complete separation assuming the null hypothesis of no difference
between groups is $2$ chances in ${10 \choose 5} = 252$ or $2/252 = 0.00794.$
wilcox.test(x1, x2)

        Wilcoxon rank sum test

data:  x1 and x2
W = 0, p-value = 0.007937
alternative hypothesis: 
   true location shift is not equal to 0

From what you say about your data, I suppose you have a complete
separation. That provides clear evidence that something has
produced a significant effect. 
However, if 'treatment' runs always followed 'control' runs (or always preceded them), then you need to mention the that the
difference might somehow be partially due to order of scanning
in addition to your 'treatment'. It's a good idea to get statistical
advice about experimental design, including randomization, before
you begin to collect data.
