I would like to perform a Mann-Whitney U Test (also called Wilcoxon rank-sum test) on a weighted sample in R. Such a non-parametric test is required, as neither of the two variables used follow normal distribution. The sample is weighted: a variable assigns a given weight to each row. The weights are numbers with decimals.

The built-in wilcox.test argument in R does not take weights into account. The '[survey]' package does offer a Wilcoxon test for weighted data but I am puzzled by the “degree of freedom” value I get upon performing it. Here is an example, with data formatted like my actual data:

ordinal = c(4, 1, 1, 2, 3, 6, 5, 7, 6, 1) #outcome variable: ordinal variable with 7 levels
groups = c(1, 1, 2, 2, 2, 2, 2, 1, 1, 2) #groups variable: factor with 2 levels
w = c(1.3, 1.3, 0.7, 0.5, 1.5, 1.6, 1.6, 0.4, 0.4, 0.7) #weights
data = data.frame(ordinal, groups, w)
data$groups<-as.factor(data$groups)....sd <- svydesign(ids=~1, probs=data$w, data=data) #survey design, used to apply weights to test
svyranktest(ordinal~groups, sd, test="wilcoxon")

Test result is displayed as follows:

Design-based KruskalWallis test

data:  ordinal ~ groups

t = -2.5834, df = 8, p-value = 0.03244

alternative hypothesis: true difference in mean rank score is not equal to 0

sample estimates:

difference in mean rank score 

Does it make sense that the test’s degree of freedom equals to 8? Or should it rather equal to 1, i.e. the amount of groups minus 1?

In a Kruskal-Wallis test, the degree of freedom is the amount of groups minus 1. I would have expected the degree of freedom in the Mann-Whitney test to be calculated the same way, as both test are very like-minded.

Yet, in this discussion thread, someone says "I suppose you could say the sample sizes are the 'degrees of freedom'", but I am not sure whether this applies to rank-sum tests (independent variables) and/or to signed-rank tests (paired variables).

From what I have also read on the internet, I get a sense that “degrees of freedom” don’t mean much when applied to a rank-sum test, yet I would really like to know whether the test is correct the way I perform it.

  • $\begingroup$ @Gautier Jacquemain Two questions: Are you comparing Group = 1 to Group = 2 and if so, why aren't you using the Mann-Whitney test? $\endgroup$ – akash87 Jun 29 '16 at 19:15
  • $\begingroup$ Also,@ChaseGrimm is correct $\endgroup$ – akash87 Jun 29 '16 at 19:16
  • $\begingroup$ @ ChaseGrimm @akash87 Thanks for your feedback, I am going to post this question on cross-validated. $\endgroup$ – Gautier Jacquemain Jun 29 '16 at 19:58
  • 2
    $\begingroup$ @akash87 Thanks for your suggestion. Yes, I am trying to compare Group = 1 with Group =2. It seems to me that a "Wilcoxon rank-sum test" and a "Mann-Whitney U test", also called "Wilcoxon-Mann-Whitney U test", are actually the same test. In that respect, the so-called "Wilcoxon test" in the "survey" package should be relevant, as it doesn't pair variables. $\endgroup$ – Gautier Jacquemain Jun 29 '16 at 20:04
  • 3
    $\begingroup$ You might do better by using the generalization of the Wilcoxon test - the proportional odds ordinal logistic model - and using case weights in that model. $\endgroup$ – Frank Harrell Apr 4 '18 at 12:08

For this design-based Wilcoxon (rank-sum) test, the degrees of freedom are the design degrees of freedom: the number of primary sampling units in the design minus the number of strata. These are denominator degrees of freedom for a $t$ or $F$ distribution. In this case, with two groups, it's a $t$ distribution. (With more than two groups there would also be a numerator degrees of freedom for an $F$ distribution, which would be one less than the number of groups; this numerator df is analogous to the df for the $\chi^2$ approximation to the Kruskal-Wallis test)

The use of design degrees of freedom in a central-limit-theorem approximation is very common in survey analysis. Simulations in the paper proposing these design-based rank tests (Lumley, T., & Scott, A. J. (2013). Two-sample rank tests under complex sampling. BIOMETRIKA, 100 (4), 831-842.) show that using the t distribution and approximate degrees of freedom gives better performance.

In this particular case the primary sampling units are the individual observations, and there is only one stratum, so the df is 8-1=7.

I should note that whether this test is what you want depends on what your weights are. svyranktest is for sampling weights, and it compares estimated population ranks rather than comparing sample ranks giving some observations more weight. The test is not exact in small samples.

The data and weights given look unusual for a probability sample -- at the very least, the weights must have been scaled, since sampling weights must be 1 or greater.

| cite | improve this answer | |

The Wilcoxon signed rank test actually does not require a degrees of freedom. In essence, the Wilcoxon signed rank test is evaluating whether or not the median of the differences is equal to 0, so this allows us to use the Central Limit Theorem and a z-score for the test statistic. Using the normal distribution gets rid of the need for a df. That being said, I am not sure why R is returning a df for this test.

| cite | improve this answer | |
  • 2
    $\begingroup$ The OP is about using case weights in Wilcoxon's rank sum test. I don't see any hint about this in your answer. $\endgroup$ – Michael M May 6 '18 at 8:30
  • 1
    $\begingroup$ This is the Kruskal-Wallis test. The "degrees of freedom" refer to a $\chi^2$ approximation to the distribution of the test statistic as explained in the Wikipedia link. $\endgroup$ – whuber Aug 21 '18 at 21:38
  • $\begingroup$ No, not in this case. The degrees of freedom are denominator degrees of freedom and refer to a $t$-approximation to the distribution of the test statistic. This is a design-based rank test for survey data, not the ordinary Wilcoxon or Kruskal-Wallis test $\endgroup$ – Thomas Lumley Jan 7 at 0:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.