I am comparing two non-normal distributions of continuous variables of a repeated-measures design (the dependent variables are reaction-times in miliseconds when reading two types of words). Therefore, I am using a Wilcoxon Signed-Rank Test, but the same phenomenon happens if I run a t-test.
If I randomly re-sort one of the distributions being compared, the test (Z or t) and p-value change.
For instance: Variable1 (n=723 of which 176 are missing data/outliers): 546, 345, 987, 1023, ...
Variable2 (N = 723 of which 102 are missing data/outliers): 567, 435, 230, 765, ...
If I randomly sort the order of the data of Variable2 so it is ordered differently, for instance: Variable2_random: 765, 567, 435, 230, ...
I get different Z/t scores and p-values when doing the test for Variable1 vs. Variable2 than for Variable1 vs Variable2_random.
Why is this? (Specifically for the Wilcoxon signed test).
If the order of data points matters for this test, in what order should I be placing the data in the data set?