1
$\begingroup$

I wish to see whether the average meal masses of two species are significantly different, and I have sample sizes of >800 for both. My Wilcoxon rank sum test returns a W = 330520 value - how should I interpret such a high value?

$\endgroup$
  • $\begingroup$ 1. How large that is depends on which of several possible forms of the Wilcoxon you're looking at; in large samples it's approximately normal with a mean and variance that depends on the sample sizes and the particular form of the statistic you used (Note that the answer here points out that values for the Mann-Whitney (an equivalent test) may be quite large in large samples.) 2. You may find it easier to interpret either as a Z-score or as an estimated P(X>Y) $\endgroup$ – Glen_b -Reinstate Monica Jun 5 '18 at 1:14
  • $\begingroup$ See also the discussion here, which may help you clarify your question. $\endgroup$ – Glen_b -Reinstate Monica Jun 5 '18 at 1:37
3
$\begingroup$

If you want to better understand what this value means, you inevitably need to understand how this value is computed.

TL;DR

Wilcoxon rank-sum test computes number of events $X > Y$

nSamples <- 100
samplesGroup1 <- rnorm(nSamples, mean = 2)
samplesGroup2 <- rnorm(nSamples, mean = -1, sd = 3)

theW <- sum(outer(samplesGroup1, samplesGroup2, '>'))
wilcox.test(samplesGroup1, samplesGroup2)$statistic == theW # = TRUE

For a large sample size, e.g. $800$, the maximal value of the statistic is $800^2$. If $W=330520$, it means that $330520/800^2$ of 'greater' comparisons is true. That is, $P(X>Y)$ ~ 50% and the two distributions are kind of indistinguishable on ordinal scale.

Longer version

Wilcoxon rank-sum test is used on samples to compare whether their distribution differ. Please make sure that your experiment, according to the assumptions, contains independent samples.

E.g. say we have following data:

library(tidyverse)

nSamples <- 100
samplesGroup1 <- rnorm(nSamples, mean = 2)
samplesGroup2 <- rnorm(nSamples, mean = -1, sd = 3)

sampleDF <- bind_rows(
  data_frame(group = 'gr1', value = samplesGroup1),
  data_frame(group = 'gr2', value = samplesGroup2)
)

To compute the statistic we need to assure assumption 2: the responses are ordinal. Therefore we transform all values of samples to ordinal scale (rank), not per each group but as a whole!

rankedDF <- sampleDF %>%
  mutate(rankedValue = rank(value)) %>%
  arrange(rankedValue) %>% # this is important for plotting
  select(- value) %>%
  group_by(group) %>%
  mutate(id = 1:n()) %>%
  spread(key = 'group', value = 'rankedValue')

Wilcoxon static computes number of events where values from one group are greater than values from another group. That involves comparing every value of one group to every values of another group. (We can use R's outer function to do exactly this).

 outer(samplesGroup1, samplesGroup2, '>')

will yield a matrix (number of samples in group 1 x number of samples in group 2) of TRUE and FALSE, where TRUE indicates that value in group 1 is greater than another value in group 2.

Visually it would look like this for 100 samples per group:

group 1 vs group 2

expand.grid(g1 = 1:nrow(rankedDF), 
            g2 = 1:nrow(rankedDF)) %>%
  # mutate(greater = rankedDF$gr1[g1] > rankedDF$gr2[g2]) %>%
  mutate(greater = as.vector(outer(rankedDF$gr1,
                                   rankedDF$gr2, ">"))) %>%
  ggplot(aes(x = g1, y = g2, fill = greater)) +
  geom_tile(color = "black") + 
  theme_bw()

Now if you count the TRUEs, i.e. sum(outer(samplesGroup1, samplesGroup2, '>')), this will be the W-statistic.

That should answer your question: a high number is due to the large sample size of >800.

To dig a little deeper, how can you interpret this number? Well, if you heard about the area under the curve, that is exactly what we see and can compute from the W-stastistic by dividing by the number of comparisons (i.e. number of squares in the plot).

$\endgroup$
  • $\begingroup$ I don't particularly like the statement of assumptions in the Wikipedia article linked to here. (I know, why don't you just edit it then, Sal?) The independence assumption is a little mushy. Usually it's stated that there must be independence of observations within the group and mutual independence of observations between the groups. Also, the statement of the null hypothesis: Under the null hypothesis H0, the distributions of both populations are equal seems off to me. Can't I come up with different distributions with stochastic equality that satisfy the null, or have I misunderstood? $\endgroup$ – Sal Mangiafico Jul 20 '18 at 13:47
  • $\begingroup$ IMHO, independence here just means the samples are not paired. Otherwise you would need to compute a different value. As for the distribution difference, you are right: you can have many distinguished shapes where AUC can be 0.5 and thus you wouldn't reject H0, although distinguished shapes would differ considerably. $\endgroup$ – Drey Jul 20 '18 at 13:56
  • $\begingroup$ The assumptions as per Conover, 1999, are copied here. The pairing issue is one, but I assume also non-independence within group would be problematic, such as serial autocorrelation, etc. $\endgroup$ – Sal Mangiafico Jul 20 '18 at 14:04
  • $\begingroup$ @SalMangiafico I think independence within group, like autocorrelation, is 'negligible' for the hypothesis that samples from one distribution are higher than from another. (Especially, if i think right, ranking introduces autocorrelation anyway). $\endgroup$ – Drey Jul 21 '18 at 4:27
1
$\begingroup$

You might consider two statistics that can be derived from the Wilcoxon rank sum test and will be easily interpretable by your audience.

The first is the p-value.

The second, as @Glen_b mentions in the comments to the question, is the probability that an observation in one group is likely to be larger than an observation from the other group. This is often called Vargha and Delaney's A, and they provide interpretations as to whether one might consider these probabilities "small", "medium", or "large", though of course these interpretations aren't universal across disciplines and specific cirumstances. VDA is closely, and linearly, related to Cliff's delta. †

If is fine, of course, to also report your sample size and W value, but your audience will be more familiar with p-value and VDA. (Although be sure to explain what VDA means, because it's not that common).


Edit: A few additional comments on VDA. Since VDA is the probability of an observation in one group being greater than an observation in the other group, a VDA of 0.50 represents no effect. VDA values closer to 1 correspond to a high probability that observations in one group are greater than the other, while values closer to 0 correspond to observations in the the other group being greater.

Assuming that tied values are accounted for properly:

1) VDA (A, B) + VDA (B, A) = 1

2) Cliff's delta, or dominance statistic, is delta = VDA(A, B) - VDA(B, A). It ranges from -1 to 1, with 0 being no effect.

3) Odds ratio = VDA (A, B) / VDA (B, A). (Or often, its inverse; usually whichever is larger.)

(See, Grisson and Kim, 2012, Effect Sizes for Research, 2nd, Chapter 5.)

$\endgroup$
  • $\begingroup$ VDA interpretation, for those without access to the article: > 0.00 – 0.29 Large; > 0.29 – 0.44 Medium; > 0.44 – < 0.56 Small; 0.56 – < 0.71 Medium; 0.71 – 1.00 Large. $\endgroup$ – Sal Mangiafico Jul 19 '18 at 14:08
  • $\begingroup$ (+1) This measure, which I often use and like a lot, goes back long before these scholars, although they clearly did a great deal of good work. Whether it's in the original Wilcoxon or Mann and Whitney papers I can't say without re-reading them, but Z.W. Birnbaum 1956 projecteuclid.org/download/pdf_1/euclid.bsmsp/1200501643 springs to mind. He called it "not new". $\endgroup$ – Nick Cox Jul 19 '18 at 21:35
  • $\begingroup$ See also this paper from 2002: stata-journal.com/sjpdf.html?articlenum=st0007 $\endgroup$ – Nick Cox Jul 19 '18 at 21:37
  • $\begingroup$ @NickCox, it's all about the marketing. You'll never sell a thing called "Probability of one thing greater than another thing." Call it Zaphod's zed, and you might have a best seller $\endgroup$ – Sal Mangiafico Jul 19 '18 at 22:18
  • $\begingroup$ Quite so. Box plots were called dispersion diagrams from 1933. Dull and dreary descriptive stuff. Come along John W. Tukey and box plots sound neat and simple. (He did have new ideas on what they should show.) $\endgroup$ – Nick Cox Jul 19 '18 at 22:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.