Here's the output from my the summary of lmer function which I used to calculate the ICC.

   Linear mixed model fit by REML ['lmerMod']
   Formula: CareChange ~ 1 + (1 | PROVIDER)
      Data: MEA_data_1

   REML criterion at convergence: 35.2

   Scaled residuals: 
       Min      1Q  Median      3Q     Max 
   -0.3093 -0.2829 -0.2711 -0.2599  3.6206 

   Random effects:
    Groups            Name        Variance  Std.Dev.
    PROVIDER          (Intercept) 0.0003096 0.01759 
    Residual                      0.0660667 0.25703 
   Number of obs: 239, groups:  PROVIDER_CALENDAR, 14

   Fixed effects:
        Estimate Std. Error t value
        (Intercept)  0.07119    0.01742   4.086

   ICC = 0.0003096 /(0.0003096 +0.0660667 ) = 0.004664315

I did not find any literature that shows how to calculate the CI for this ICC value. Any help on this issue is much appreciated.

A straightforward way to calculate a confidence interval would be to create a bootstrapped distribution, then obtain the relevant quantiles from that distribution. This can be done as a parametric or nonparametric bootstrap, depending on what assumptions you're comfortable with. Generally, if you accept a Normal distribution for the random effects, a parametric bootstrap is also appropriate.

Since you're using R, the lme4 package offers a very nice parametric bootstrap function, bootMer() to calculate a bootstrapped distribution of any statistic of interest derived from a random effects model. The parametric method is fully implemented, which simulates the model of interest using new values of the random effects with each iteration. New values are drawn from a Normal distribution using parameters derived from the mixed model. See the documentation in bootMer() for some detail on control over what effects get bootstrapped. Set your own number of iterations, but 1000+ is a good rule of thumb.

To calculate this in R, you need to fit the random effects model and pass a function to bootMer() that calculates the statistic of interest. An example implementation is below:

Example Code using R

#Make some mocked data
library(lme4)
library(reshape2)
set.seed(2024)  #For the Bell Riots
id <- factor(seq(1, 15))
id.mu <- rnorm(15, 10, 5)
mydat <- NULL
for (a in 1:length(id)){
  score <- rnorm(2, id.mu[a], 3)
  id.fr <- data.frame(id=id[a], score1=score[1], score2=score[2])
  mydat <- rbind(mydat, id.fr)
}
mydat <- melt(mydat, id.vars='id', value.name='score')

#Create function to calculate ICC from fitted model
calc.icc <- function(y) {
  sumy <- summary(y)
  (sumy$varcor$id[1]) / (sumy$varcor$id[1] + sumy$sigma^2)
}

#Fit the random effects model and calculate the ICC
mymod <- lmer(score ~ 1 + (1|id), data=mydat)
summary(mymod)
calc.icc(mymod)

#Calculate the bootstrap distribution
boot.icc <- bootMer(mymod, calc.icc, nsim=1000)

#Draw from the bootstrap distribution the usual 95% upper and lower confidence limits
quantile(boot.icc$t, c(0.025, 0.975))

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.