0
$\begingroup$

I am comparing two non-normal distributions of continuous variables of a repeated-measures design (the dependent variables are reaction-times in miliseconds when reading two types of words). Therefore, I am using a Wilcoxon Signed-Rank Test, but the same phenomenon happens if I run a t-test.

If I randomly re-sort one of the distributions being compared, the test (Z or t) and p-value change.

For instance: Variable1 (n=723 of which 176 are missing data/outliers): 546, 345, 987, 1023, ...

Variable2 (N = 723 of which 102 are missing data/outliers): 567, 435, 230, 765, ...

If I randomly sort the order of the data of Variable2 so it is ordered differently, for instance: Variable2_random: 765, 567, 435, 230, ...

I get different Z/t scores and p-values when doing the test for Variable1 vs. Variable2 than for Variable1 vs Variable2_random.

Why is this? (Specifically for the Wilcoxon signed test).

If the order of data points matters for this test, in what order should I be placing the data in the data set?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
3
$\begingroup$

Repeat measures are paired. If the pairing (i.e., rank order) is randomized, then paired difference data tests like Wilcoxon signed rank testing are invalid. This is because the order of ranking is used to find a correlation, and if the order is mixed up, the correlation is destroyed.

Improve this answer
$\endgroup$
8
  • $\begingroup$ Thank you! However, I still don´t know in which order to insert the responses in the data set. $\endgroup$
    – Daniel Low
    Oct 22, 2016 at 17:53
  • $\begingroup$ Each subject read words from two conditions in a random order. Therefore, each data point isn´t exactly paired to another data point from the other condition. Furthermore, both variables have different amount of missing data. $\endgroup$
    – Daniel Low
    Oct 22, 2016 at 17:57
  • $\begingroup$ Are you sorting responses to each word type for each subject? Is each word type tested more than once for each subject? $\endgroup$
    – Carl
    Oct 23, 2016 at 13:29
  • $\begingroup$ Answer to first question: I am not inserting responses in my data set in the order they were recorded, but grouping them into their respective condition. Second question: yes. Subjects read 24 different words from condition 1 and 24 different words from condition 2 in random order (e.g., cond2 cond1 1 2 2 1 2 NA 2...), the same random order for all participants. Should I list the final data set in the order the responses were recorded (participant by participant) or can I not use Wilcoxon for this comparison? $\endgroup$
    – Daniel Low
    Oct 23, 2016 at 15:30
  • $\begingroup$ What is your hypothesis? $\endgroup$
    – Carl
    Oct 23, 2016 at 16:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.