1
$\begingroup$

In brief I'm trying to find out if 62 people perform better on test one or test two.

I ran a Spearman's rho and it has come back positive and significant with a moderate strength. I.E As the test 1 scores increased so did test 2 scores increase.

However when I ran the same data through a Wilcoxon matched pairs it has come back significant but showing that the ranked means and medians were lower for the second test than for the first. I have checked the data 3 times and it always comes back the same.

Can someone help me understand why this might be happening please?

| cite | improve this question | | | | |
$\endgroup$
2
$\begingroup$

There's no contradiction whatever. Those outcomes are perfectly consistent with each other.

Populations might differ in location (one way of getting a good chance of rejection in the Wilcoxon signed rank test) while being correlated (leading to likely rejection in the test of Spearman's rho).

The significant result for the correlation measure tells you nothing about the relative size of the values on the two tests, only that they tend to move in the same direction (broadly, the same people tend to do well or poorly on both)

Imagine for example that the scores in group 2 were always about 5 points lower on test 2. Then people who scored highest on test 1 would also score among the highest on test 2 and people who scored lowest on test 1 would also score among the lowest on test 2 -- the (Spearman*) correlation would be very close to 1. But the median of the differences would still be around 5. Both p-values would be about as small as they could be.

* or indeed any other common measure of correlation.

enter image description here

showing that the [...] medians were lower for the second test than for the first.

Actually rejection in the Wilcoxon signed rank test doesn't necessarily indicate that. If the assumption of symmetry holds, this should tend to be the case*, but in general rejection by the Wilcoxon signed rank test doesn't imply a difference in population medians is non-zero but rather that population pseudo-median of the differences** is nonzero.

* 1. In a symmetric population the pseudomedian is same as the median but in a sample they can still differ. 2. You're talking about the difference in medians rather than the median of the differences, in general all three things may be different in populations or in samples.

**(the population quantity corresponding to the Hodges-Lehmann estimator)

The population pseudo-median is the median of pairwise averages among all possible pairs in the population. It's sometimes said to be "like the median" but the two can be different (and can differ in sign).

Here's an example illustrating that the median difference and the difference in medians needn't even have the same sign:

![enter image description here

but even further, because the pesudomedian of the differences is 0 here, if I add or subtract a small amount (say 0.01) to (/from) each z2 value, the Hodges-Lehmann estimate of location of the pair-differences can be made positive or negative (without changing the other two quantities by very much, so they retain their present signs). As a result the estimate of location difference in the Wilcoxon signed rank test can differ in sign from either the median of the differences or from the difference in medians.

In short, if you hold that the assumption of symmetry in the population of pair-differences is true, then a non-zero population psuedomedian of pair-differences implies a non-zero population median of pair-differences, but beware of referring to that as a difference in population medians.

| cite | improve this answer | | | | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.