Does the ICC require that my data are normally distributed?

If more information about my data is needed, just ask.

  • $\begingroup$ The problem this user has is a similar problem im facing. It relates to the factthat an f test is done $\endgroup$ – Cesare Camestre Jul 19 '13 at 6:25
  • 1
    $\begingroup$ Answeredj in stats.stackexchange.com/questions/3539/… - the answer is "yes", if you want to use the standard errors and F tests spat out by a default stats routine (which is implied in the OP and in @Cesare's comments). Better to use a bootstrap for any inference. $\endgroup$ – Peter Ellis Jul 21 '13 at 3:37
  • $\begingroup$ One can calculate a weighted kappa coefficient for ordinal data that wouldn't require a normal distribution, although this does require some assumptions regarding the weighting scheme used. I added a comment to Peter's answer to the linked question which has a few more details (@PeterEllis) $\endgroup$ – James Stanley Jul 23 '13 at 0:28

Depends on the method - if you're using the ANOVA (general linear) method, then yes. But you can also calculate the ICC using generalized linear methods (I'm not sure about non-parametric methods - anyone else?).

Nakagawa and Schielzeth wrote the user-friendly rptR package for R, which uses general or generalized linear mixed effects modelling (for normal, binomial and count data) to calculate repeatability within classes using ANOVA, REML or MCMC methods. I think this is what you want to do?


  • $\begingroup$ What about stata? $\endgroup$ – Cesare Camestre Jul 19 '13 at 7:19
  • $\begingroup$ No, i'm after some measure of inter-rater agreement, for 21 respondents who rated something from 0-7. $\endgroup$ – Cesare Camestre Jul 19 '13 at 7:31
  • $\begingroup$ So you're not interested in whether a certain respondents consistently rate things higher or lower than other respondents? That's what I think when I hear inter-rater agreement. Or if you're only interested in how much the 21 respondents agree on a particular question, what's wrong with just using the standard deviation of their ratings for that question? I don't use Stata, sorry, but having browsed the documentation, it looks like it only supports the F-test, which would require normal data? $\endgroup$ – atrichornis Jul 19 '13 at 8:32
  • $\begingroup$ I'm interested in whether the respondents rated the same items the same so.. say I have 7 items in a list, the respondents agree in the way they rated these items. Was there general agreement that the item should be rated high? $\endgroup$ – Cesare Camestre Jul 19 '13 at 9:23
  • 1
    $\begingroup$ Oh okay, my apologies. I gather ICC might not be the best measure in this case - there are other measures of inter-rater agreement that are better for ordinal data, e.g. Kappa. People with more knowledge than me have answered similar questions elsewhere on here: e.g. stats.stackexchange.com/questions/3539/… $\endgroup$ – atrichornis Jul 19 '13 at 11:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.