I want to calculate the ICC between 3 different measurements where the dependent variable is a count. As far as I understood, if the data were normally distributed, I would use a repeated measures ANOVA for this. But for count data, the normality assumption doesn't hold true. Do you have any hints how to get the ICC without using rmANOVA? I know that generalized linear models are somehow crucial for that, but I don't know how to exactly use them for this purpose.

  • $\begingroup$ Welcome to the site. This is a good question. I have edited it for clarity; please ensure it still says what you want it to. Eg, I changed the 'independent variable is a count' to the dependent variable. Note that the distribution of your IV is irrelevant, so I suspect you meant DV. If that's wrong please let us know. $\endgroup$ – gung - Reinstate Monica Sep 23 '15 at 15:00

You need to work with a Generalized Linear Mixed-effects Model (GLMM). The repeated measures ANOVA is actually a special case of the Linear Mixed-effects Model (LMM), so GLMM:LMM as GLM:LM. From there, just recognize that the ICC is a descriptive statistic that assesses how distinctive the units are relative to the total spread of the data. The standard formula is:
$$ ICC = \frac{\sigma_{\bar{x_i}}^2}{\sigma_{\bar{x_i}}^2+\sigma_\varepsilon^2} $$ In the context of a mixed-effects model, the distinctiveness of the individuals is the variance of the random effects. Since variances add, the total is the variance of the random effects plus the residual variance.

To illustrate, here is a simple example, coded in R:

library(lme4)   # we'll use this package to fit the GLMM
set.seed(1189)  # this makes the example exactly reproducible

N          = 20                                     # there are 20 people in your study
reps       = 3                                      # w/ 3 observation each
id         = rep(1:N, each=reps)                    # this is the ID indicator
sub.lambda = rnorm(N, mean=1.6, sd=.3)              # here we generate each unit's mean
sub.lambda = rep(sub.lambda, each=reps)             #  & copy them for each observation
y          = rpois(reps*N, lambda=exp(sub.lambda))  # here we generate the counts

mod = glmer(y~(1|id), family=poisson)               # this fits the Poisson GLMM
summary(mod)                                        # here is the output:
# Generalized linear mixed model fit by maximum likelihood 
#   (Laplace Approximation) [glmerMod]
#  Family: poisson  ( log )
# ... 
# Random effects:
#   Groups Name        Variance Std.Dev.
#   id     (Intercept) 0.1066   0.3265  
# Number of obs: 60, groups:  id, 20
# Fixed effects:
#             Estimate Std. Error z value Pr(>|z|)    
# (Intercept)  1.45853    0.09745   14.97   <2e-16 ***

  ## now we get the variance of the random effects, the residuals, & calculate the ICC:
v.RE  = summary(mod)$varcor$id[1];  v.RE   # 0.1066211
v.res = var(resid(mod));            v.res  # 1.141968
ICC   = v.RE/(v.RE+v.res);          ICC    # 0.08539324
  • $\begingroup$ Thanks a lot for your answer. I recently also read that a Friedman's test could be used for this purpose. What do you think of it? And does this GLMM also work for incomplete data, means NAs in the dataset or unequal number of observations per measurement? $\endgroup$ – soschnei Sep 27 '15 at 9:37
  • $\begingroup$ @soschnei, I would not use Friedman't test to determine the ICC here, but you could use it for a test of a treatment effect. If your dataset is naturally unbalanced (you simply took measures at different times), there is no problem. If you have missing data (you should have had data there) a GLMM is fine as long as the data are missing at random (MAR). $\endgroup$ – gung - Reinstate Monica Sep 27 '15 at 10:43
  • $\begingroup$ Thanks again. I have some more questions: Why didn't you use the number of repetitions as a fixed effect? I found quite some models in which days of repetitions or comparable units were used as a fixed effect. This gives me very different results. And if I would have an additional factor, e.g. I measures counts for left and right leg, how would you include it in the model? Also using a random effect? And if I use the mean counts instead of raw count data, I would still have to use glme, wouldn't I? The data is not integer anymore but is also not allowed to be negative. $\endgroup$ – soschnei Oct 7 '15 at 11:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.