1
$\begingroup$

In my research, I use a questionnaire so that students mention some classmates according to a group of characteristics. Each item is a different characteristic, and for each one they must name the classmate they consider to have it. In this case can I use Krippendorff's alpha to assess the reliability of the students' judgments?

$\endgroup$

1 Answer 1

1
$\begingroup$

Krippendorff's alpha is a measure of (chance corrected) agreement. Reliability in classical test theory is the relationship between the true score and the measured score. However, measures such as Kripp alpha are often referred to as reliability (but maybe I'm just old).

However, you have a weird issue, that people (presumably) cannot nominate themselves, so 100% agreement isn't possible.

$\endgroup$
2
  • $\begingroup$ This means that in the first place one could not speak of reliability, since I am not trying to measure any construct, but rather the nomination of people who have a group of characteristics is sought. In this way, you could obtain the % of agreement between the participants, however this could not be 100% because they could not self-nominate. Is this the idea? $\endgroup$ Jul 26, 2022 at 15:02
  • $\begingroup$ Yeah, alpha wouldn't be horrible, just be aware it's going to be a little lower than it could be. $\endgroup$ Jul 26, 2022 at 16:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.