We have two sets of raters judge the same items before and after a training, resulting in an ICC and kappa value for each time point (pre- and post-training). How would you statistically compare pre- and post-training reliability?
For ICC - is it valid to merely perform a paired-samples t-test? I've read that bootstrapping is an option, but I don't understand the rationale behind it or methods to achieve this.
