1
$\begingroup$

I have noted the differences between the tests, as answered here (Difference between the Wilcoxon Rank Sum Test and the Wilcoxon Signed Rank Test).

However, I would like to know whether there are any tendencies for over/under-estimations of the test statistic if e.g. wilcox.test(.., paired=TRUE) is used instead of wilcox.test(.., paired=FALSE) (or the other way around) and how large that effect could be as a function of sample size. How would you estimate or express this without testing?

Presume that we have equal sample size in the two groups and that no exclusion of uneven pairs is needed and disregard the fact that one of the method is correct for the data. I just want to be able to mathematically estimate this effect/error it might cause in terms of test statistic.

$\endgroup$
1
$\begingroup$

Data is either paired or not. Consequently, it is either appropriate to use the paired test$-$Wilcoxon signed-rank W test, or the unpaired test$-$Mann-Whitney U test (A.K.A. Wilcoxon rank sum test), but never both. It is possible to ignore pairing and erroneously use the less powerful rank sum test. That has the tendency of spuriously increasing the probability of significant results and decreasing the probability of highly insignificant results, and although that might have meaning in some other context, e.g., contrasting Wilcoxon with t-testing for example, it has no meaning here.

It is difficult to imagine a common scenario in which one would mistakenly use the paired test on unpaired data as typically the groups contrasted have unequal sizes. Frankly I do not know what would happen if the sizes were equal and one mistakenly used the paired test. It is somewhat pointless to ask as well, as one should not be doing that.

If one wants to see effect size despite having a motive for NOT doing so, i.e., it is an error, one might think that one could do a Monte Carlo simulation to explore it. My personal problem in doing so is that if I start with a false premise, the result of such a simulation would not be logical, which makes it conceptually difficult for me to evolve appropriate other assumptions to use to set up that simulation. Perhaps that is only my personal problem, so I ask, and what exactly does one assume to set up such a test? For example, randomly correlated pairing? If so, randomly correlated how? Should the correlation itself be modeled as random, or should it be random correlation that arises from random data?

$\endgroup$
  • $\begingroup$ I think that either you misunderstood me or the question was vaguely asked. See edit. $\endgroup$ – nadizan Mar 5 '18 at 13:43
  • $\begingroup$ A better question would ask about effect size for Wilcoxon, paired or unpaired versus $t$-testing, paired or unpaired, respectively for distributions that are 1) normal 2) quasi-normal 3) not normal. There is no useful effect size for an error, so what is the motive here? $\endgroup$ – Carl Mar 5 '18 at 19:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.