I'm currently calculating an intraclass correlation coefficient for my data. As far as I'm concerned, the best choice (following this approach and the approach kindly recommended on the previous post answer) is:
Model: 2-way random effects ## two teachers' assessment of the same group of students on an exam (it could be any other pair of teachers randomly selected, thus I believe it's generalizable)
Type: average ## we'll be using the two-teacher's average in further analysis
Definition: absolute agreement
### check if variance is homogeneous: car::leveneTest(data$RATER1 ~ data$RATER2) ##I don't know if it's necessary, but I've seen that ICC relies on the assumption that variance is homogeneous ### run it: library(irr) icc(data[,2:3], model = "twoway", type = "agreement", unit = "average")
That being said, I've seen elsewhere some comments on using only 20 to 25% of the sample to calculate ICC, such as in this article and in another internet post, but not nowhere else. Why is that?
- 1 Is it ok to use the whole sample or Should I only perform it with 20 to 25% of the data? Thanks in advance.