1
$\begingroup$

When running Cohen's Kappa in SPSS, it outputs a few things.

  1. Cohen's Kappa
  2. Asymptotic Standard Error (ASE)
  3. Approximate T
  4. Approximate Significance

If I have a Kappa of 0.90 and ASE of .012, can I determine my 95% confidence interval as $k=0.90 \pm 1.96*.012$?

Secondly, the significance and relationship to Kappa. If my $p =0.01$, then am I saying the Almost perfect agreement between raters is significant?

$\endgroup$
  • $\begingroup$ I formatted the plus minus for you. The only snag I can see with your proposed method is that it could lead to intervals outside the permissible range for $\kappa$ $\endgroup$ – mdewey Oct 20 '18 at 14:12
  • $\begingroup$ These were totally made up samples so....given that if we did have a better range, then I guess it is alright. $\endgroup$ – Jack Armstrong Oct 20 '18 at 16:07
1
$\begingroup$

The early development of Cohen's $\kappa$ gave rise to several attempts to estimate the standard error of $\kappa$. The issue was finally resolved in a paper by Fleiss and colleagues entitled "Large sample standard errors of kappa and weighted kappa" available here in which they give formulas for unweighted and weighted $\kappa$ both for the case where the hypothesised value is zero (hence appropriate for testing that hypothesis) and for any value (hence appropriate for establishing a confidence interval about the estimated value). Note that these are not the same.

Which one SPSS computes is not clear although the formula is not that complicated so it could be checked by hand if the documentation is not explicit. Note that testing the hypothesis $\kappa=0$ is usually superfluous as if there is a serious possibility that it might be zero the agreement must be trivially small.

$\endgroup$
0
$\begingroup$

I believe the null hypothesis of the test is the k is not different from zero, while alternative is that k is different from zero. Therefore, a p = 0.01 indicates that your k is different from zero. The degree of agreement has to be judged from your k.

Please correct me if I am mistaken.

$\endgroup$
  • $\begingroup$ I would have upvoted if your 1st sentence had been more precise and consistent with the rest.... $\endgroup$ – rolando2 Oct 20 '18 at 20:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.