2
$\begingroup$

I have two series of binary outcomes:

      Series1 Series2
Test1 0       1
Test2 1       1
Test3 1       1
Test4 0       1
Test5 0       0
...

I want to know whether or not the outcomes in the two series were significantly different.

Can I use the Wilcoxon signed rank test for this purpose?

I.e., in R:

wilcox.test(data1, data2, paired=TRUE)

Improve this question
$\endgroup$
2
  • $\begingroup$ You could (you type the command, it happens), but should not. The test is designed for a continuous outcome. There are other tests for the binary case. $\endgroup$
    – Glen_b
    Commented Mar 26, 2019 at 23:49
  • $\begingroup$ What about when one series is binary and the other is continuous? $\endgroup$
    – BigMistake
    Commented Oct 2 at 18:22

1 Answer 1

Reset to default
2
$\begingroup$

For binary and independent variables you should use a chi-square test if the Central Limit Theorem's assumptions are not violated or Fisher's exact test if they are.

Here is a piece of code that tests the CLT assumptions and runs the relevant test depending on the outcome.

I've used a rule of thumb value of 5. Feel free to change my number "5" with something a higher value for more conservative or less if you want to be less conservative:

# Generate some random binary outcomes for Series 1 and 2
set.seed(1241535)
Series1 <- rbinom(150, 1, 0.5)
Series2 <- rbinom(200, 1, 0.6)

# All tests need to be confirmed
pooled_p <- (sum(Series1) + sum(Series2))/(length(Series1) + length(Series2))

test1 <- (length(Series1) * pooled_p) >= 5
test2 <- (length(Series2) * pooled_p) >= 5
test3 <- (length(Series1) * (1 - pooled_p)) >= 5
test4 <- (length(Series2) * (1 - pooled_p)) >= 5

final_test <- all(test1, test2, test3, test4)

# Chi-square or Fisher's exact test
x    <- c(sum(Series1), sum(Series2))
n    <- c(length(Series1), length(Series2))
mash <- rbind(c(sum(Series1), length(Series1) - sum(Series1)),
              c(sum(Series2), length(Series2) - sum(Series2)))

if(final_test == T){

  ## With Yate's continuity correction

  prop.test(x,n)
  #Exactly the same as:
  chisq.test(mash)

}else{

  # Fisher's exact test
  fisher.test(mash)

}

If your variables are not independent e.g: if series 1 and 2 are measurements of the same individual before and after an intervention, then a McNemar's test is more appropriate:

set.seed(1241535)
Series1 <- rbinom(200, 1, 0.5)
Series2 <- rbinom(200, 1, 0.6)


tab <-
  matrix(c(sum(Series1 == 1 & Series2 == 1), 
           sum(Series1 == 0 & Series2 == 1), 
           sum(Series1 == 1 & Series2 == 0), 
           sum(Series1 == 0 & Series2 == 0)
           ),
         nrow = 2,
         dimnames = list("Series1" = c("1", "0"),
                         "Series2" = c("1", "0")))

tab
mcnemar.test(tab)

The latter might be the relevant one for your case as I've noticed that you specify paired = TRUE in your code.

Improve this answer
$\endgroup$
7
  • $\begingroup$ Take a look at what the Central limit theorem actually says. Could you explain which of the assumptions of the central limit theorem (or indeed any of the central limit theorems, since there are several) are relevant to choosing whether to do a chi-squared test or something else? $\endgroup$
    – Glen_b
    Commented Mar 26, 2019 at 23:53
  • $\begingroup$ @Glen_b Oh you mean that the outcomes of series 1 and 2 might not be independent? You are right, I will edit my answer to account for that case as well. Thanks for your correction, it was my bad to assume independence. $\endgroup$
    – Vasilis Vasileiou
    Commented Mar 27, 2019 at 0:51
  • 1
    $\begingroup$ No. I meant simply that I can see nothing in the central limit theorem itself that says anything about how to choose when the chi-squared approximation is not suitable. Your first sentence mentions "central limit theorem assumptions". Which assumption violations would make the chi-squared unsuitable but leave the Fisher exact test unaffected? $\endgroup$
    – Glen_b
    Commented Mar 27, 2019 at 2:24
  • $\begingroup$ The sample size. In the case of independent samples and when we want to assess the frequency difference H0:p1=p2 vs H2:Otherwise, the statistic that we derive is Z = ((p1hat - p2hat) - (p1 - p2))/(SE(p1_hat-p2_hat)) which under the null hypothesis of p1=p1=p becomes Z = (p1hat - p2hat) /(sqrt(p(1-p)(1/n1 + 1/n2))). The derived test statistic converges in distribution to N(0,1). This is what I meant by assumptions of CLT, that we basically need sufficiently large samples for this convergence to be valid. en.wikipedia.org/wiki/Pearson%27s_chi-squared_test#Assumptions $\endgroup$
    – Vasilis Vasileiou
    Commented Mar 27, 2019 at 8:53
  • 1
    $\begingroup$ Fixed that. Name of the corrected table is "tab" for future readers $\endgroup$
    – Vasilis Vasileiou
    Commented Mar 30, 2019 at 15:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.