Paired or unpaired non-parametric 2 sample test?

Question

I've got a relative simple question. I'm doing an imaging study with a phantom. I've run the same phantom through the MRI scanner 5 times, each time running the same scan twice and only varying one parameter. I am trying to assess if varying this parameter (independent variable) changes my response variable (dependent variable). Do I assume this is a paired measurement because both tests were conducted sequentially in the same session? I get a p value of 0.0625 running the Wilcoxon matched-pairs signed rank test (a function of a low sample number I assume), vs 0.008 when the Mann-Whitney U test is run. Thanks for your help!

Answer 1

A paired approach seems appropriate because of the way data were collected. A paired Wilcoxon test is essentially the same a one-sample Wilcoxon Signed Rank test on the differences.

Unfortunately, assuming no ties among the five differences, the smallest possible P-value $1/16 = 0.0625$ for a 2-sided test occurs when all differences have the same sign. (By chance alone there is 1 chance in 32 that all five differences would be positive and the same that tall five would be negative: $1/32+1/32 = 1/16.)$

Here are computations in R statistical software, based on some hypothetical data:

x1 = c(1,4,5,7,8)
x2 = c(12,18,17,20,23)
d = x2-x1; d
[1] 11 14 12 13 15

Identical P-values of $1/16$ for both versions of the Wilcoxon signed-rank test, using paired data:

wilcox.test(x1, x2, pair=T)

        Wilcoxon signed rank test

data:  x1 and x2
V = 0, p-value = 0.0625
alternative hypothesis: 
   true location shift is not equal to 0

wilcox.test(d)

        Wilcoxon signed rank test

data:  d
V = 15, p-value = 0.0625
alternative hypothesis: 
    true location is not equal to 0

Doing a 2-sided Mann-Whitney-Wilcoxon two-sample rank sum test, there are ten observations altogether, then if (as for my illustrative hypothetical data) all observations in one group are larger than any value in the other a smaller P-value is possible. Specifically, the P-value for a complete separation assuming the null hypothesis of no difference between groups is $2$ chances in ${10 \choose 5} = 252$ or $2/252 = 0.00794.$

wilcox.test(x1, x2)

        Wilcoxon rank sum test

data:  x1 and x2
W = 0, p-value = 0.007937
alternative hypothesis: 
   true location shift is not equal to 0

From what you say about your data, I suppose you have a complete separation. That provides clear evidence that something has produced a significant effect.

However, if 'treatment' runs always followed 'control' runs (or always preceded them), then you need to mention the that the difference might somehow be partially due to order of scanning in addition to your 'treatment'. It's a good idea to get statistical advice about experimental design, including randomization, before you begin to collect data.