9
$\begingroup$

One of the assumptions for using the Wilcoxon sign-rank test is that the underlying distribution is continuous (see here.)

However, there are cases (for example, when analyzing Likert scale data) where this assumption might not necessarily hold. In such cases, what test can you use? And how would you do it with R?

(My only bet here is to use a randomization test on the median - which I imagine can be easily done using the boot package.)

Improve this question
$\endgroup$
4
  • $\begingroup$ Likert scale is actually a good example of an underlying continuous variable that is collected in discrete fashion. $\endgroup$
    – Aniko
    Commented Nov 23, 2010 at 19:38
  • $\begingroup$ @Aniko Right, but the problem is that we're not sure how well the original interval scale fits into the discretized one, unless making strong assumptions. Optimal scaling might be an option there. $\endgroup$
    – chl
    Commented Nov 23, 2010 at 21:13
  • $\begingroup$ @chl That's why you are using Wilcoxon's test instead of a t-test. Wilcoxon's test does not assume equally spaced intervals, etc, so scaling is not an issue. $\endgroup$
    – Aniko
    Commented Nov 23, 2010 at 22:11
  • $\begingroup$ @Aniko I agree with you. My comment was not a critic, and reference to optimal scaling was mainly for the case where the assumption of an a priori continuous scale of measurement is not tenable--because in practice, we often rely on Likert items without making strong assumption on the underlying construct; I also agree for Mann-Whitney test with ordinal variables. $\endgroup$
    – chl
    Commented Nov 24, 2010 at 11:28

1 Answer 1

Reset to default
8
$\begingroup$

I have found that the Wilcoxon statistic is still fine for this purpose and that small simulations do a good job of estimating the size and the power of the test. I suspect this is more powerful than just comparing the two medians. The main concern is lack of power due to extensive numbers of ties, but that concern attaches to any solution you can conceive of: there's no way around it (except to design instruments that offer a wider range of responses!).

To perform the simulation, concatenate the two data arrays (of lengths $n$ and $m$) into a single array (of length $n+m$). In each iteration randomly permute the elements of the array and break the result into the first $n$ and last $m$ elements.

Improve this answer
$\endgroup$
8
  • 2
    $\begingroup$ (+1) Agree, i think the assumption of a continuous distribution may affect the standard way of calculating p-values, but simulation as suggested by whuber gets around that (i'd call this a permutation test). $\endgroup$
    – onestop
    Commented Nov 23, 2010 at 20:19
  • 5
    $\begingroup$ @onestop That is correct. Indeed, the Wilcoxon test is a permutation test. All we're discussing is how to compute (or estimate) the distribution of the test statistic. The distribution can be computed exactly using combinatorial techniques; it can be estimated through simulation; and it can be approximated, usually with a Normal approximation when the amount of data is large enough (m+n of 10 or greater is usually good enough) and the data can be thought of as realizations of a continuous RV. $\endgroup$
    – whuber ♦
    Commented Nov 23, 2010 at 20:38
  • $\begingroup$ You're right of course @whuber, i meant "i'd call that a Monte-Carlo permutation test*". (Sorry was in too much of a hurry for no good reason really) $\endgroup$
    – onestop
    Commented Nov 23, 2010 at 22:58
  • $\begingroup$ @onestop I didn't mean to sound critical. I was agreeing with you and felt your meaning was clear. $\endgroup$
    – whuber ♦
    Commented Nov 23, 2010 at 23:07
  • 1
    $\begingroup$ @DWin For a continuous underlying distribution and sufficient sample sizes it is approximately normally distributed. The exact distribution, however, is not normal and departs appreciably from normality when there are less than 10 values (in both datasets combined). Tables of the distribution have been published to cover those small-data cases, but it is readily computed nowadays by brute force. (E.g., with 10 values it takes little time to generate all the possible combinations.) $\endgroup$
    – whuber ♦
    Commented Nov 25, 2010 at 2:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.