I have been tasked to approximate the error rate of a research procedure.
The only data I have available are reports which specify the number of targets (n), the number of raters (k) and give pairwise inter-rater reliability (Fleiss' kappa) values. The 'true values' of the targets are unknown, as are the values that raters assigned to individual targets.
Is there a way to estimate how many errors each single rater might have made based on that information?
I came across these posts:
But I am unsure whether they address the issue I am facing. Any hints appreciated!