# Validity of grand mean intra-rater Kappa or ICC

For 5 raters, I have calculated intra-rater Cohen's Kappa statistics for the test-retest nominal ratings, intra-rater Kendall's tau-b statistics for the test-retest ordinal ratings, and intra-rater ICCs for the test-retest scale ratings. Is it valid to calculate overall mean intra-rater reliability coefficients (Kappa, Kendall's tau-b, ICC, as appropriate) across all raters for each of these variables? If so, can I simply average the individual raters' reliability coefficients for overall means?

• The average of such reliability coefficients is, just that, the average of such reliability coefficients. Why would it be interesting or useful? Best to design an analysis of all kinds of variation, between and within raters. – Nick Cox Jun 4 '13 at 18:18