5
$\begingroup$

I'm interested in meta-analysing the agreement between two measures. Depending on whether the variables are continuous/categorical/binary, different studies present agreement as either:

  • A Pearson's correlation coefficient (for continuous variables)
  • Intraclass correlation coefficient/weighted kappa (for categorical/ordinal variables)
  • Cohen's kappa (for binary variables)

I understand that intraclass correlation coefficient is equivalent to a weighted kappa under certain conditions, which increases the comparability between the ICC and kappa.

Is there a way to convert a Pearson's correlation coefficient to an equivalent of Cohen's kappa?

Improve this question
$\endgroup$
1
  • 4
    $\begingroup$ Even data are just binary (so correlation becomes the phi coefficient), correlation is not comparable with kappa. They differ in the denominator. So, if you don't have the 2x2 table itself, from which these associations are computed, you're at a deadlock. $\endgroup$
    – ttnphns
    Commented Feb 4, 2021 at 17:02

1 Answer 1

Reset to default
6
$\begingroup$

Consider the following scenario

One rater rates the objects as 1,2,3 Second rater rates them 3,4,5

The correlation is 1 but the value of unweighted kappa is -0.12 and weighted (with quadratic weights) is 0.25

So, in general, they are not even close.

Credits: analysis done with function cohen.kappa from the R psych package and the cor function from R stats

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.