I have a list of N people, with two equivalent test performed (A and B), which can have as outcome 0 or 1 (cancer positive or negative). Test A is the accepted gold standard.

Is there any way to find if there are some kind of correlation between both? I found some solutions based on $\chi^2$, but I don't think that can be applied to this case. I also found some related solutions based on ROC curve analysis, but I think is far more complicated than what I need it.

  • 1
    $\begingroup$ I think this question would benefit from a more specific title - perhaps "measuring inter-rater reliability with binary data" or similar? $\endgroup$
    – Silverfish
    Commented Apr 15, 2016 at 10:04

3 Answers 3


Related to you might also want to take a look into the Phi coefficient:

, where is the total number of observations. This coefficient is specifically designed for binary variables and the logic behind it is that in the contingency table behind the binary variables:

    B1   B0
 A1 n11  n10
 A0 n01  n00

, your data will be more "correlated" if it falls under the n11->n00 diagonal. There are other similarity coefficients you might want to consider such as Jaccard Index.

In here you'll get a comprehensive list of some other possibilities.

Measures for Binary Data


If you want a correlation specifically, the tetrachoric correlation coefficient is meant for comparing two binary distributions (to estimate it you should have at least 50 observations).

You can calculate the tetrachoric correlation coefficient in R using the polychor package:

polychor(data, ML=T, std.err=T)

Alternatively, the $\phi$ coefficient as suggested by @armatita is also the binary form of the Pearson product-moment correlation coefficient. An alternate formulation from the $\chi^2$ one is the following:

$$ \phi=\sqrt{\frac{\chi^2}{n}}=\frac{ad-bc}{\sqrt{(a+b)(a+c)(b+d)(c+d)}} $$

where $a,b,c,d$ are defined by the following contingency table for tests $A$ and $B$:

$$ \begin{array} ~&A=1&A=0\\ B=1&a&b\\ B=0&c&d\\ \end{array} $$


The normal test for association in binary data would be χ2, and as far as I can see this would seem to deliver the analysis you want. You may wish to undertake subsidiary analysis, including on any disagreement between tests A and B, but χ2 would be a reasonable first step. [There are other tests you could apply, but χ2 will do the job adequately, is simple and is well-understood.]

  • 2
    $\begingroup$ Thanks for the comment. Using a $\chi^2$ test, I need to make a contingency table and then will I not lose the fact that are paired observations? $\endgroup$
    – Greynes
    Commented Apr 15, 2016 at 8:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.