-1
$\begingroup$

I have 260 teacher conferences to score using a rubric. How do two raters need to score these for rater agreement? Do we need three raters?

$\endgroup$
0
$\begingroup$

The use of a rubric is there to try to control for inter-rater variability. You can further decrease inter-rater variability by 'training' the raters (e.g. show all the raters the same video and tell them what they should score the video). Of course, in terms of getting a more precise estimate of the true rating, the more raters the better

$\endgroup$
0
$\begingroup$

With 2 raters only you have no way of knowing which rater is more accurate. With 3 raters or more you can use factor analysis to account for raters precision.

$\endgroup$
  • $\begingroup$ He could't know which is more accurate, but he could still measure their concordance to some extent, could he not? $\endgroup$ – Behacad Feb 7 '15 at 4:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.