I have two series of binary outcomes:

      Series1 Series2
Test1 0       1
Test2 1       1
Test3 1       1
Test4 0       1
Test5 0       0

I want to know whether or not the outcomes in the two series were significantly different.

Can I use the Wilcoxon signed rank test for this purpose?

I.e., in R:

wilcox.test(data1, data2, paired=TRUE)

  • $\begingroup$ You could (you type the command, it happens), but should not. The test is designed for a continuous outcome. There are other tests for the binary case. $\endgroup$
    – Glen_b
    Mar 26, 2019 at 23:49

1 Answer 1


For binary and independent variables you should use a chi-square test if the Central Limit Theorem's assumptions are not violated or Fisher's exact test if they are.

Here is a piece of code that tests the CLT assumptions and runs the relevant test depending on the outcome.

I've used a rule of thumb value of 5. Feel free to change my number "5" with something a higher value for more conservative or less if you want to be less conservative:

# Generate some random binary outcomes for Series 1 and 2
Series1 <- rbinom(150, 1, 0.5)
Series2 <- rbinom(200, 1, 0.6)

# All tests need to be confirmed
pooled_p <- (sum(Series1) + sum(Series2))/(length(Series1) + length(Series2))

test1 <- (length(Series1) * pooled_p) >= 5
test2 <- (length(Series2) * pooled_p) >= 5
test3 <- (length(Series1) * (1 - pooled_p)) >= 5
test4 <- (length(Series2) * (1 - pooled_p)) >= 5

final_test <- all(test1, test2, test3, test4)

# Chi-square or Fisher's exact test
x    <- c(sum(Series1), sum(Series2))
n    <- c(length(Series1), length(Series2))
mash <- rbind(c(sum(Series1), length(Series1) - sum(Series1)),
              c(sum(Series2), length(Series2) - sum(Series2)))

if(final_test == T){

  ## With Yate's continuity correction

  #Exactly the same as:


  # Fisher's exact test


If your variables are not independent e.g: if series 1 and 2 are measurements of the same individual before and after an intervention, then a McNemar's test is more appropriate:

Series1 <- rbinom(200, 1, 0.5)
Series2 <- rbinom(200, 1, 0.6)

tab <-
  matrix(c(sum(Series1 == 1 & Series2 == 1), 
           sum(Series1 == 0 & Series2 == 1), 
           sum(Series1 == 1 & Series2 == 0), 
           sum(Series1 == 0 & Series2 == 0)
         nrow = 2,
         dimnames = list("Series1" = c("1", "0"),
                         "Series2" = c("1", "0")))


The latter might be the relevant one for your case as I've noticed that you specify paired = TRUE in your code.

  • $\begingroup$ Take a look at what the Central limit theorem actually says. Could you explain which of the assumptions of the central limit theorem (or indeed any of the central limit theorems, since there are several) are relevant to choosing whether to do a chi-squared test or something else? $\endgroup$
    – Glen_b
    Mar 26, 2019 at 23:53
  • $\begingroup$ @Glen_b Oh you mean that the outcomes of series 1 and 2 might not be independent? You are right, I will edit my answer to account for that case as well. Thanks for your correction, it was my bad to assume independence. $\endgroup$ Mar 27, 2019 at 0:51
  • 1
    $\begingroup$ No. I meant simply that I can see nothing in the central limit theorem itself that says anything about how to choose when the chi-squared approximation is not suitable. Your first sentence mentions "central limit theorem assumptions". Which assumption violations would make the chi-squared unsuitable but leave the Fisher exact test unaffected? $\endgroup$
    – Glen_b
    Mar 27, 2019 at 2:24
  • $\begingroup$ The sample size. In the case of independent samples and when we want to assess the frequency difference H0:p1=p2 vs H2:Otherwise, the statistic that we derive is Z = ((p1hat - p2hat) - (p1 - p2))/(SE(p1_hat-p2_hat)) which under the null hypothesis of p1=p1=p becomes Z = (p1hat - p2hat) /(sqrt(p(1-p)(1/n1 + 1/n2))). The derived test statistic converges in distribution to N(0,1). This is what I meant by assumptions of CLT, that we basically need sufficiently large samples for this convergence to be valid. en.wikipedia.org/wiki/Pearson%27s_chi-squared_test#Assumptions $\endgroup$ Mar 27, 2019 at 8:53
  • 1
    $\begingroup$ Fixed that. Name of the corrected table is "tab" for future readers $\endgroup$ Mar 30, 2019 at 15:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.