1
$\begingroup$

This question already has an answer here:

I'm learning statistics but don't really understand the result of kappa value posted in the first sample here.

Is there anyone help me explain in English what the value 0.915 means? (observed proportionate agreement) and the kappa value 0.801. Does 0.915 is the value that both A and B agree with the totally 94 proposals? Or something?

$\endgroup$

marked as duplicate by kjetil b halvorsen, usεr11852, John, mdewey, gung Mar 20 '17 at 12:42

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

  • $\begingroup$ Welcome to Cross Validated! Please take a moment to view our tour. This appears to be a self-study question. If that is the case, please read our wiki and tag your question as well. $\endgroup$ – Tavrock Mar 14 '17 at 9:58
  • $\begingroup$ Thank you for your suggestion. I just added "self-study" tag. $\endgroup$ – Peter Sm Mar 14 '17 at 10:04
  • $\begingroup$ @JeffreyGirard This is asking about the meaning of various Kappa values, which although a simple question, is not covered in the link to another question provided. $\endgroup$ – Carl Mar 19 '17 at 18:55
  • 1
    $\begingroup$ @usεr11852 I suppose, but I don't like the answers there either; too much wind for too little in the sails. If you want to close this question, I think my simple answer would need porting over, see below. $\endgroup$ – Carl Mar 19 '17 at 22:10
  • 1
    $\begingroup$ @usεr11852 Oh, bother. So fully go there and vote please: stats.meta.stackexchange.com/questions/4690/… $\endgroup$ – Carl Mar 19 '17 at 23:16
1
$\begingroup$

$$\kappa=\frac{p_o-p_c}{1-p_c}$$ where $p_o$ is the percent observed agreement and $p_c$ is the percent chance agreement (estimated).

The interpretation of a kappa coefficient (Cohen's or otherwise) is the amount of observed non-chance agreement divided by the possible amount of non-chance agreement. So if you correctly classified 80% of the items, but we would expect you to have gotten 20% of them correct by chance alone, then you would have a kappa value of $0.75$ which is smaller than observed agreement (i.e., 80%).

$$\kappa=\frac{0.80-0.20}{1-0.20}=\frac{0.60}{0.80}=0.75$$

$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.