Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

I found a question that is related here, but it doesn't really goes on what I want to know.

I found a couple of papers using Kappa Statistic from 2006, and 2010, but afterwards I found other authors disagreeing with it like here and here. I also noticed other tests being listed and suggested like the ICC on the question pointed on cross validated I pointed, and on the wiki.

To narrow down the context: I have a set n of participants which consistently were asked to say 'yes/no/maybe' on a set of questions. I've seen authors also changing the yes/maybe to 'evidence' and converting it to yes/no by just summing the quantities of yes and maybe.

My questions thus are:

  1. What is the most recent/accepted or referenced paper in respect to this discussion?
  2. Given my dataset constraints, which test would be more appropriated? I was considering Fleiss but when I saw it was a generalization of the pi-statistic and that the Kappa came out of criticism to pi i got confused. If you could suggest an R function that could do it I would also appreciate.

Thank you.

share|improve this question
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.