I'm analysing data from a paper in which 4 raters; medical professionals, estimate the volume of an organ, they do this twice, for every organ. I want to compute inter-rater and intra-agreement on the results; there are six equations to compute ICC (as listed in Wikipedia):
Shrout and Fleiss Name in SPSS convention ICC(1,1) One-way random single measures ICC(1,k) One-way random average measures ICC(2,1) Two-way random single measures (Consistency/Absolute agreement) ICC(2,k) Two-way random average measures (Consistency/Absolute agreement) ICC(3,1) Two-way mixed single measures (Consistency/Absolute agreement) ICC(3,k) Two-way mixed average measures (Consistency/Absolute agreement)
I don't know whether I want ICC(2,1) or ICC(3,1) - the question seems to be 'are these the only raters of interest' or 'were they chosen randomly'? This detail isn't in the paper; another set of medical professionals with relevant expertise could also have done the rating, the most likely scenario is that these 4 were the ones who agreed first to help in the study.