# Should we test only 20% of the sample for intraclass correlation coefficient?

I'm currently calculating an intraclass correlation coefficient for my data. As far as I'm concerned, the best choice (following this approach and the approach kindly recommended on the previous post answer) is:

Model: 2-way random effects ## two teachers' assessment of the same group of students on an exam (it could be any other pair of teachers randomly selected, thus I believe it's generalizable)
Type: average ## we'll be using the two-teacher's average in further analysis
Definition: absolute agreement

• code:
### check if variance is homogeneous:
car::leveneTest(data$$RATER1 ~ data$$RATER2) ##I don't know if it's necessary, but I've seen that ICC relies on the assumption that variance is homogeneous

### run it:
library(irr)
icc(data[,2:3], model = "twoway", type = "agreement", unit = "average")


That being said, I've seen elsewhere some comments on using only 20 to 25% of the sample to calculate ICC, such as in this article and in another internet post, but not nowhere else. Why is that?

Question:

• 1 Is it ok to use the whole sample or Should I only perform it with 20 to 25% of the data? Thanks in advance.

Preamble

In the article you cite (Tong et al. 2022), production recordings codings (i.e., the DV) were coded by either author 2 or author 3 (i.e., each production recording was initially coded by author 1). Because the authors are interested in calculating rater agreement, such a data set is not sufficient because each production recording was only rated once by one of the two coders. Instead of simply having each coder code all production recordings (which I assume would take a considerable amount of time), the authors instead just randomly selected 25% of the production recordings to be coded by two coders (i.e., author 1 and either author 2 or author 3).